This paper introduces a new process, environment matting ,w hich captures not just a foreground object and its traditional opacity matte from a real-world scene, but also a description of how that ob- ject refracts and reflects light, which we call an environment matte. The foreground object can then be placed in a new environment, us- ing environment compositing, where it will refract and reflect light from that scene. Objects captured in this way exhibit not only spec- ular but glossy and translucent effects, as well as selective attenua- tion and scattering of light according to wavelength. Moreover, the environment compositing process, which can be performed largely with texture mapping operations, is fast enough to run at interactive speeds on a desktop PC. We compare our results to photos of the same objects in real scenes. Applications of this work include the relighting of objects for virtual and augmented reality, more realis- tic 3D clip art, and interactive lighting design.