Self-Supervised Multi-Label Classification with Global Context and Local Attention

Self-supervised learning has proven highly effective across various tasks, showcasing its versatility in different applications. Despite these achievements, the challenges inherent in multi-label classification have seen limited attention. This paper introduces GAELLE, a novel self-supervised multi-label classification framework that simultaneously captures image context and object information. GAELLE employs a combination of global context and local attention mechanisms to discern diverse levels of semantic information in images. The global component comprehensively learns image content while local attention eliminates object-irrelevant nuances by aligning embeddings with a projection head. The integration of global and local features in GAELLE effectively captures intricate object-scene relationships. To further enhance this capability, we introduce a global and local swap prediction technique, facilitating the nuanced interplay between various objects and scenes within images. Experimental results showcase GAELLE’s state-of-the-art performance in self-supervised multi-label classification tasks, highlighting its effectiveness in uncovering complex relationships between multiple objects and scenes in images.