Proliferation of woody plants in grasslands and savannas is a persistent problem globally. This widely observed shift from grass to shrub dominance in rangelands worldwide has been heterogeneous in space and time largely due to cross‐scale interactions among soils, climate, and land‐use history. Our objective was to use a hierarchical framework to evaluate the relationship between spatial patterns in soil properties and long‐term shrub dynamics in the northern Chihuahuan Desert of New Mexico, USA. To meet this objective, shrub patch dynamics from 1937 to 2008 were characterized at patch and landscape scales using historical imagery and a recent digital soils map. Effects of annual precipitation on patch dynamics on two soils revealed strong correlations between shrub growth on deep sandy soils and above‐average rainfall years (r = 0.671, P = 0.034) and shrub colonization and below‐average rainfall years on shallow sandy soils (r = 0.705, P = 0.023). Patch‐level analysis of demographic patterns revealed significant differences between shrub patches on deep and shallow sandy soils during periods of above‐ and below‐average rainfall. Both deep and shallow sandy soils exhibited low shrub cover in 1937 (1.0% ± 2.3% and 0.3% ± 1.3%, respectively [mean ± SD]) and were characterized by colonization or appearance of new patches until 1960. However, different demographic responses to the cessation of severe drought on the two soils and increased frequency of wet years after 1960 have resulted in very different endpoints. In 2008 a shrubland occupied the deep sandy soils with cover at 19.8% ± 9.1%, while a shrub‐dominated grassland occurred on the shallow sandy soils with cover at 9.3% ± 7.2%. Present‐day shrub vegetation constitutes a shifting mosaic marked by the coexistence of patches at different stages of development. Management implications of this long‐term multi‐scale assessment of vegetation dynamics support the notion that soil properties may constrain grassland remediation. Such efforts on sandy soils should be focused on sites characterized by near‐surface water‐holding capacity, as those lacking available water‐holding capacity in the shallow root zone pose challenges to grass recovery and survival.