Neutral atoms are ideal objects for the deterministic processing of quantum information. Entanglement operations have been performed by photon exchange or controlled collisions. Atom-photon interfaces were realized with single atoms in free space or strongly coupled to an optical cavity. A long standing challenge with neutral atoms, however, is to overcome the limited observation time. Without exception, quantum effects appeared only after ensemble averaging. Here we report on a single-photon source with one-and-only-one atom quasi permanently coupled to a high-finesse cavity. Quasi permanent refers to our ability to keep the atom long enough to, first, quantify the photon-emission statistics and, second, guarantee the subsequent performance as a single-photon server delivering up to 300,000 photons for up to 30 seconds. This is achieved by a unique combination of single-photon generation and atom cooling. Our scheme brings truly deterministic protocols of quantum information science with light and matter within reach.