An inconsistency previously found between the relativistic and non-relativistic second-order hyperfine energies of the hydrogen atom with a point nucleus is ascribed to the neglect of delta function contributions in the non-relativistic calculation. Various relations involving point delta functions are developed. In particular, a novel representation of the commutator of partial differential operators acting on a homogeneous function of degree −n + 2 in n variables is given in terms of the point delta function. With the inclusion of the delta function terms, the non-relativistic point-nucleus result is shown to be consistent with the results of both relativistic point-nucleus and non-relativistic finite-nucleus calculations.