Energy Moments of Scattering Phase Shifts II. Higher Partial Waves