The Optics of Polarization Sensitivity