The visual microphone
Top Cited Papers
Open Access
- 27 July 2014
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 33 (4), 1-10
- https://doi.org/10.1145/2601097.2601119
Abstract
When sound hits an object, it causes small vibrations of the object's surface. We show how, using only high-speed video of the object, we can extract those minute vibrations and partially recover the sound that produced them, allowing us to turn everyday objects---a glass of water, a potted plant, a box of tissues, or a bag of chips---into visual microphones. We recover sounds from high-speed footage of a variety of objects with different properties, and use both real and simulated data to examine some of the factors that affect our ability to visually recover sound. We evaluate the quality of recovered sounds using intelligibility and SNR metrics and provide input and recovered audio samples for direct comparison. We also explore how to leverage the rolling shutter in regular consumer cameras to recover audio from standard frame-rate videos, and use the spatial resolution of our method to visualize how sound-related vibrations vary over an object's surface, which we can use to recover the vibration modes of an object.Keywords
Funding Information
- Qatar Foundation
- National Science Foundation (CGV-1111415, 1122374)
- Massachusetts Institute of Technology
- Microsoft Research
This publication has 19 references indexed in Scilit:
- Simultaneous remote extraction of multiple speech sources and heart beats from secondary speckles patternOptics Express, 2009
- Content-preserving warps for 3D video stabilizationACM Transactions on Graphics, 2009
- New Image Processing Tools for Structural Dynamic MonitoringKey Engineering Materials, 2007
- Motion magnificationACM Transactions on Graphics, 2005
- Interactive digital photomontageACM Transactions on Graphics, 2004
- What energy functions can be minimized via graph cuts?IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- A phase-based approach to the estimation of the optical flow field using spatial filteringIEEE Transactions on Neural Networks, 2002
- Shiftable multiscale transformsIEEE Transactions on Information Theory, 1992
- Laser vibrometry: Pseudo-vibrationsJournal of Sound and Vibration, 1989
- Resonances of a Violin Body Studied by Hologram Interferometry and Acoustical MethodsPhysica Scripta, 1970