This article reviews developments in the theory of rapidly rotating degenerate atomic gases. The main focus is on the equilibrium properties of a single component atomic Bose gas, which (at least at rest) forms a Bose-Einstein condensate. Rotation leads to the formation of quantized vortices which order into a vortex array, in close analogy with the behaviour of superfluid helium. Under conditions of rapid rotation, when the vortex density becomes large, atomic Bose gases offer the possibility to explore the physics of quantized vortices in novel parameter regimes. First, there is an interesting regime in which the vortices become sufficiently dense that their cores -- as set by the healing length -- start to overlap. In this regime, the theoretical description simplifies, allowing a reduction to single particle states in the lowest Landau level. Second, one can envisage entering a regime of very high vortex density, when the number of vortices becomes comparable to the number of particles in the gas. In this regime, theory predicts the appearance of a series of strongly correlated phases, which can be viewed as {\it bosonic} versions of fractional quantum Hall states. This article describes the equilibrium properties of rapidly rotating atomic Bose gases in both the mean-field and the strongly correlated regimes, and related theoretical developments for Bose gases in lattices, for multi-component Bose gases, and for atomic Fermi gases. The current experimental situation and outlook for the future are discussed in the light of these theoretical developments.