There is considerable uncertainty in the magnitude and direction of changes in precipitation associated with climate change, and ecosystem responses are also uncertain. Multiyear periods of above- and below-average rainfall may foretell consequences of changes in rainfall regime. We compiled long-term aboveground net primary productivity (ANPP) and precipitation (PPT) data for eight North American grasslands, and quantified relationships between ANPP and PPT at each site, and in 1-3 year periods of above- and below-average rainfall for mesic, semiarid cool, and semiarid warm grassland types. Our objective was to improve understanding of ANPP dynamics associated with changing climatic conditions by contrasting PPT-ANPP relationships in above- and below-average PPT years to those that occurred during sequences of multiple above- and below-average years. We found differences in PPT-ANPP relationships in above- and below-average years compared to long-term site averages, and variation in ANPP not explained by PPT totals that likely are attributed to legacy effects. The correlation between ANPP and current- and prior-year conditions changed from year to year throughout multiyear periods, with some legacy effects declining, and new responses emerging. Thus, ANPP in a given year was influenced by sequences of conditions that varied across grassland types and climates. Most importantly, the influence of prior-year ANPP often increased with the length of multiyear periods, whereas the influence of the amount of current-year PPT declined. Although the mechanisms by which a directional change in the frequency of above- and below-average years imposes a persistent change in grassland ANPP require further investigation, our results emphasize the importance of legacy effects on productivity for sequences of above- vs. below-average years, and illustrate the utility of long-term data to examine these patterns.