Auto-annotation quality prediction for semi-supervised learning with ensembles

Auto-annotation by an ensemble of models is an efficient method of learning on unlabeled data. However, wrong or inaccurate annotations generated by the ensemble may lead to performance degradation of the trained model. We propose filtering the auto-labeled data using a trained model that predicts the quality of the annotation from the degree of consensus between ensemble models. Using semantic segmentation as an example, we demonstrate the advantage of the proposed auto-annotation filtering over training on data contaminated with inaccurate labels. We show that the performance of a state-of-the-art model can be achieved by training it with only a fraction (30%) of the original manually labeled samples, and replacing the rest with auto-annotated, quality filtered labels.