We analyze the problem of determining the electronic ground state within O(N) schemes, focusing on methods in which the total energy is minimized with respect to the density matrix. We note that in such methods a crucially important constraint is that the density matrix must be idempotent (i.e. its eigenvalues must all be zero or unity). Working within orthogonal tight-binding theory, we analyze two related methods for imposing this constraint: the iterative purification strategy of McWeeny, as modified by Palser and Manolopoulos; and the minimization technique of Li, Nunes and Vanderbilt. Our analysis indicates that the two methods have complementary strengths and weaknesses, and leads us to propose that a hybrid of the two methods should be more effective than either method by itself. This idea is tested by using tight-binding theory to apply the proposed hybrid method to a set of condensed matter systems of increasing difficulty, ranging from bulk crystalline C and Si to liquid Si, and the effectiveness of the method is confirmed. The implications of our findings for O(N) implementations of non-orthogonal tight-binding theory and density functional theory are discussed.