The Helmholtz Machine
- 1 September 1995
- journal article
- Published by MIT Press in Neural Computation
- Vol. 7 (5), 889-904
- https://doi.org/10.1162/neco.1995.7.5.889
Abstract
Discovering the structure inherent in a set of patterns is a fundamental aim of statistical inference or learning. One fruitful approach is to build a parameterized stochastic generative model, independent draws from which are likely to produce the patterns. For all but the simplest generative models, each pattern can be generated in exponentially many ways. It is thus intractable to adjust the parameters to maximize the probability of the observed patterns. We describe a way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations. Our method can be viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.Keywords
This publication has 11 references indexed in Scilit:
- A Learning Algorithm for Boltzmann Machines*Published by Wiley ,2010
- The "Wake-Sleep" Algorithm for Unsupervised Neural NetworksScience, 1995
- Competition and Multiple Cause ModelsNeural Computation, 1995
- A Multiple Cause Mixture Model for Unsupervised LearningNeural Computation, 1995
- A Bayesian Analysis of Self-Organizing MapsNeural Computation, 1994
- A forward-inverse optics model of reciprocal connections between visual cortical areasNetwork: Computation in Neural Systems, 1993
- Forward models: Supervised learning with a distal teacherCognitive Science, 1992
- Connectionist learning of belief networksArtificial Intelligence, 1992
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992
- Pattern-recognizing stochastic learning automataIEEE Transactions on Systems, Man, and Cybernetics, 1985