Excess phase noise has been observed in microwave kinetic inductance detectors (MKIDs) which prevents the noise-equivalent power (NEP) of current detectors from reaching theoretical limits. One characteristic of this excess noise is its dependence on the power of the readout signal: the phase noise decreases as the readout power increases. We investigated this power dependence in a variety of devices, varying the substrate (silicon and sapphire), superconductor (aluminum and niobium) and resonator parameters (resonant frequency, quality factor and resonator geometry). We find that the phase noise has a power law dependence on the readout power, and that the exponent is -1/2 in all our devices. We suggest that this phase noise is caused by coupling between the high-Q microwave resonator that forms the sensitive element of the MKID and two-level systems associated with disorder in the dielectric material of the resonator. The physical situation is analogous to the resonance fluorescence in quantum optics, and we are investigating the application of resonance fluorescence theory to MKID phase noise.