Recover Corrupted Data in Sensor Networks: A Matrix Completion Solution
Open Access
- 29 July 2016
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 16 (5), 1434-1448
- https://doi.org/10.1109/tmc.2016.2595569
Abstract
Affected by hardware and wireless conditions in WSNs, raw sensory data usually have notable data loss and corruption. Existing studies mainly consider the interpolation of random missing data in the absence of the data corruption. There is also no strategy to handle the successive missing data. To address these problems, this paper proposes a novel approach based on matrix completion (MC) to recover the successive missing and corrupted data. By analyzing a large set of weather data collected from 196 sensors in Zhu Zhou, China, we verify that weather data have the features of low-rank, temporal stability, and spatial correlation. Moreover, from simulations on the real weather data, we also discover that successive data corruption not only seriously affects the accuracy of missing and corrupted data recovery but even pollutes the normal data when applying the matrix completion in a traditional way. Motivated by these observations, we propose a novel Principal Component Analysis (PCA)-based scheme to efficiently identify the existence of data corruption. We further propose a two-phase MC-based data recovery scheme, named MC-Two-Phase, which applies the matrix completion technique to fully exploit the inherent features of environmental data to recover the data matrix due to either data missing or corruption. Finally, the extensive simulations with real-world sensory data demonstrate that the proposed MC-Two-Phase approach can achieve very high recovery accuracy in the presence of successively missing and corrupted data.Keywords
Funding Information
- National High Technology Research and Development Program of China (863 Program) (2015AA01A705)
- Jiangsu Future Networks Innovation Institute (BY2013095-4-06)
- National Natural Science Foundation of China (61572184, 61300219, 61472283, 61271185, 61472131)
- US National Science Foundation (CNS 1526843)
This publication has 35 references indexed in Scilit:
- A Singular Value Thresholding Algorithm for Matrix CompletionSIAM Journal on Optimization, 2010
- Optimizing the Spatio-temporal Distribution of Cyber-Physical Systems for Environment AbstractionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- Fixed point and Bregman iterative methods for matrix rank minimizationMathematical Programming, 2009
- Applying PCA for Traffic Anomaly Detection: Problems and SolutionsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Security in wireless sensor networksIEEE Wireless Communications, 2008
- Sensitivity of PCA for traffic anomaly detectionPublished by Association for Computing Machinery (ACM) ,2007
- ROBPCA: A New Approach to Robust Principal Component AnalysisTechnometrics, 2005
- A first step toward understanding inter-domain routing dynamicsPublished by Association for Computing Machinery (ACM) ,2005
- Network anomographyPublished by Association for Computing Machinery (ACM) ,2005
- Nearest neighbor pattern classificationIEEE Transactions on Information Theory, 1967