In this paper, we propose photon counting three-dimensional (3D) passive sensing and object recognition using integral imaging. The application of this approach to 3D automatic target recognition (ATR) is investigated using both linear and nonlinear matched filters. We find there is significant potential of the proposed system for 3D sensing and recognition with a low number of photons. The discrimination capability of the proposed system is quantified in terms of discrimination ratio, Fisher ratio, and receiver operating characteristic (ROC) curves. To the best of our knowledge, this is the first report on photon counting 3D passive sensing and ATR with integral imaging.