Grazing can alter the spatial heterogeneity of vegetation, influencing ecosystem processes and biodiversity. Our objective was to identify why grazing causes increases in the spatial heterogeneity of vegetation in some cases, but decreases in others. The immediate effect of grazing on heterogeneity depends on the interaction between the spatial pattern of grazing and the pre-existing spatial pattern of vegetation. Depending on the scale of observation and on the factors that determine animal distribution, grazing patterns may be stronger or weaker than vegetation patterns, or may mirror the spatial structure of vegetation. For each possible interaction between these patterns, we make a prediction about resulting changes in the spatial heterogeneity of vegetation. Case studies from the literature support our predictions, although ecosystems characterized by strong plant-soil interactions present important exceptions. While the processes by which grazing causes increases in heterogeneity are clear, how grazing leads to decreases in heterogeneity is less so. To explore how grazing can consistently dampen the fine-scale spatial patterns of competing plant species, we built a cell-based simulation model that features two competing plant species, different grazing patterns, and different sources of vegetation pattern. Only the simulations that included neighborhood interactions as a source of vegetation pattern produced results consistent with the predictions we derived from the literature review.