We experimentally and theoretically investigate in-medium propagation effects of off-resonant light in dense, spatially homogeneous ultacold atomic gases. Focussing on frequency modulation spectroscopy as the dispersive detection tool of atoms, we observe that the refractive gradient-index lenses presented by localised atomic ensembles can significantly modify the interpretation of the dispersive signal even for large probe detuning, owing to the collective dispersive response of the atoms. We identify criteria for distinguishing between thin and thick atomic lenses, leading to either diffraction-dominated and lensing dominated regimes for the outgoing probe beams. Our findings are consistent with experimental data and solutions of paraxial wave equation for light propagation. Our study provides important practical insights for dispersive, minimally intrusive optical detection and imaging schemes of ultracold atoms and will be valuable for choosing optimal parameter regimes in numerous applications.