We study the effects of polarizable substrates such as SiO2 and SiC on the carrier dynamics in graphene. We find that the quasiparticle spectrum acquires a finite broadening due to the long-range coupling with the polar modes arising at the interface between graphene and the substrate. The scattering rate has a strong temperature dependence and possibly constitutes the dominant limiting factor of the mobility at room temperature, especially for SiO2 substrates. The effect of the substrate is smaller in bilayer graphene, due to screening and density of states effects.