The problems of inverting experimental information obtained from vibration-rotation spectroscopy to determine the potential energy surface of a molecule are discussed, both in relation to semi-rigid molecules like HCN, NO2, H2CO, etc., and in relation to non-rigid or floppy molecules with large amplitude vibrations like HCNO, C3O2, and small ring molecules. Although standard methods exist for making the necessary calculations in the former case, they are complex, and they require an abundance of precise data on the spectrum that is rarely available. In the case of floppy molecules there are often data available over many excited states of the large amplitude vibration, but there are difficulties in knowing the precise form of the large amplitude coordinate(s), and in allowing for the vibrational averaging effects of the other modes. In both cases difficulties arise from the curvilinear nature of the vibrational paths which are not adequately handled by our present theories.