Learning from Weak and Noisy Labels for Semantic Segmentation
- 7 April 2016
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 39 (3), 486-500
- https://doi.org/10.1109/tpami.2016.2552172
Abstract
A weakly supervised semantic segmentation (WSSS) method aims to learn a segmentation model from weak (image-level) as opposed to strong (pixel-level) labels. By avoiding the tedious pixel-level annotation process, it can exploit the unlimited supply of user-tagged images from media-sharing sites such as Flickr for large scale applications. However, these `free' tags/labels are often noisy and few existing works address the problem of learning with both weak and noisy labels. In this work, we cast the WSSS problem into a label noise reduction problem. Specifically, after segmenting each image into a set of superpixels, the weak and potentially noisy image-level labels are propagated to the superpixel level resulting in highly noisy labels; the key to semantic segmentation is thus to identify and correct the superpixel noisy labels. To this end, a novel L 1 -optimisation based sparse learning model is formulated to directly and explicitly detect noisy labels. To solve the L 1 -optimisation problem, we further develop an efficient learning algorithm by introducing an intermediate labelling variable. Extensive experiments on three benchmark datasets show that our method yields state-of-the-art results given noise-free labels, whilst significantly outperforming the existing methods when the weak labels are also noisy.Keywords
Other Versions
Funding Information
- National Natural Science Foundation of China (61573363, 61573026)
- 973 Program of China (2014CB340403, 2015CB352502)
- Fundamental Research Funds for the Central Universities and the Research Funds of Renmin University of China (15XNLQ01)
- IBM Global SUR Award Program
- European Research Council FP7 Project SUNNY (313243)
- KAUST
This publication has 45 references indexed in Scilit:
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- From image-level to pixel-level labeling with Convolutional NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Fully convolutional networks for semantic segmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Multiple Structured-Instance Learning for Semantic Segmentation with Uncertain Training DataPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- The Role of Context for Object Detection and Semantic Segmentation in the WildPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Learning Hierarchical Features for Scene LabelingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- Are spatial and global constraints really necessary for segmentation?Published by Institute of Electrical and Electronics Engineers (IEEE) ,2011
- Harmony PotentialsInternational Journal of Computer Vision, 2011
- Label to region by bi-layer sparsity priorsPublished by Association for Computing Machinery (ACM) ,2009
- Category Level Object Segmentation by Combining Bag-of-Words Models with Dirichlet Processes and Random FieldsInternational Journal of Computer Vision, 2009