We show that the Gaussian core model of particles interacting via a penetrable repulsive Gaussian potential, first considered by Stillinger (J. Chem. Phys. 65, 3968 (1976)), behaves like a weakly correlated ``mean field fluid'' over a surprisingly wide density and temperature range. In the bulk the structure of the fluid phase is accurately described by the random phase approximation for the direct correlation function, and by the more sophisticated HNC integral equation. The resulting pressure deviates very little from a simple, mean-field like, quadratic form in the density, while the low density virial expansion turns out to have an extremely small radius of convergence. Density profiles near a hard wall are also very accurately described by the corresponding mean-field free-energy functional. The binary version of the model exhibits a spinodal instability against de-mixing at high densities. Possible implications for semi-dilute polymer solutions are discussed.