The pulse height defect in semiconductor detectors