Open Access

Daniel Pototzky

• Supervised learning, the standard paradigm in machine learning, only works well if a sufficiently large, diverse, and cleanly-annotated dataset is available. Unfortunately, this is often not the case. In fact, the lack of labeled data is an omnipresent issue in machine learning. The problem is particularly prevalent in computer vision, where unlabeled images or videos can often be acquired at a low cost, whereas labeling them is time-consuming and expensive. To address the issue, this thesis focuses on developing new methods that aim at reducing annotation costs in computer vision by leveraging unlabeled and partially labeled data. In the first part, we provide an overview of previous research directions and discuss their strengths and weaknesses. Thereby, we identify particularly promising research areas. The subsequent chapters which form the central part of this thesis aim at developing algorithmic improvements in these especially attractive fields. Among them is self-supervised learning, which aims at learning transferable representations given a large number of unlabeled images. We find that existing self supervised methods are optimized for image classification tasks, only compute global per-image feature vectors, and are designed for object-centric datasets like ImageNet. To address these issues, we propose a method that is particularly suited for object detection downstream tasks and works well if multiple objects are present per image like in video data for autonomous driving. Another core downside of self-supervised learning algorithms is that they depend on very large batch sizes with batch norm statistics synchronized across GPUs and also require many epochs of training until convergence. We find that stabilizing the self-supervised training target substantially speeds up convergence and allows for training with much smaller batch sizes. Our method matches ImageNet weights after 25 epochs of training with a batch size of only 32. Finally, we investigate supervised pretraining. We find that state-of-the-art self-supervised methods match ImageNet weights only in classification or detection but not in both. In addition, we show that more sophisticated supervised training strategies significantly improve upon ImageNet weights. The second part of the thesis deals with partially labeled data for object detection. We propose to label only large, easy-to-spot objects given a limited budget. We argue that these contain more pixels and therefore usually more information about the underlying object class than small ones. At the same time, they are easier to spot and hence cheaper to label. Because conventional supervised learning algorithms do not work well given this annotation protocol, we develop our own method with does, by combining pseudo-labels, output consistency across scales, and an anchor scale-dependent ignore strategy. Furthermore, many object detection datasets such as MS COCO and CityPersons include group annotations, i.e., bounding boxes that contain multiple objects of a single class. We find that pseudo-labeling instances within a group box is superior to the commonly used training strategies. In the third part of the thesis, we cover semi-supervised object detection where a subset of the images is fully labeled whereas the remaining ones are unlabeled. We show that existing methods that are almost exclusively developed for Faster R-CNN work much less well if applied to architectures that are sensitive to missing annotations. In the prefinal chapter, we investigate the interaction between data and computer vision algorithms. This is in contrast to the vast majority of research which considers the data to be fixed. We provide computer vision practitioners and researchers with guidelines about what to do in typical situations. In the final part of the thesis, we discuss the overall findings and investigate if research should put greater weight on acquiring and labeling data. Finally, we discuss options of mimicking human learning with machines, which might eventually result in human-level intelligence. After all, humans are living proof that this kind of learning works, if done properly.