REPOSITORY > RESULTS

Abstract

The rapid expansion of Earth observation (EO) systems has generated unprecedented
volumes of satellite imagery, creating both remarkable opportunities and significant challenges
for environmental monitoring and land cover analysis. While this abundance of data
enables comprehensive global monitoring, extracting actionable insights requires sophisticated
analytical methods capable of modeling the inherent complexity of Earth’s landscapes,
where multiple land cover types and complex semantic relationships often coexist
within a single image. Traditional supervised single-label classification approaches prove
inadequate for such scenarios, necessitating structured output prediction frameworks that
can capture multi-label dependencies and taxonomic constraints while operating effectively
under limited supervision.
This dissertation addresses the critical intersection of structured output prediction and
varying levels of supervision in remote sensing image analysis. The research focuses on developing
novel semi-supervised learning methodologies for multi-label classification (MLC)
and hierarchical multi-label classification (HMLC) that effectively leverage abundant unlabeled
satellite imagery alongside limited labeled samples while preserving complex label
dependencies inherent in real-world visual content.
We develop a hybrid semi-supervised framework that combines convolutional neural
network-based feature extraction with semi-supervised predictive clustering trees (SSLPCTs)
and random forests (SSL-RForest). This two-step approach leverages the representational
power of deep learning while maintaining the flexibility of tree-based methods for
structured output prediction. Through extensive evaluation across multiple remote sensing
datasets, we demonstrate that SSL-PCT and SSL-RForest consistently outperform their
supervised counterparts across different network architectures and proportions of labeled
data, with SSL-RForest emerging as particularly effective compared to state-of-the-art selfsupervised
and semi-supervised methods. To enhance the computational efficiency of this
framework, we investigate dimensionality reduction techniques, achieving dramatic speedups
of 2-3 orders of magnitude in training times while maintaining or improving predictive
performance, making the approach more practical for large-scale applications.
Motivated by the computational inefficiencies and suboptimal feature alignment inherent
in two-stage learning approaches, we introduce SSL-MAE: a semi-supervised learning
framework for multi-label classification of remote sensing images based on masked autoencoders.
This unified end-to-end framework overcomes the limitations of traditional
separate pre-training and fine-tuning phases by integrating self-supervised representation
learning with task-specific discriminative learning through adaptive joint optimization.
Using masked autoencoders as the self-supervised component, this framework dynamically
balances supervision signals throughout training, achieving competitive performance
against two-stage methods while significantly improving computational efficiency. The
adaptive joint learning mechanism demonstrates broad applicability, successfully enhancing
other state-of-the-art self-supervised techniques, including Bootstrap Your Own Latent
(BYOL) and Momentum Contrast (MoCo). Recognizing that real-world land cover taxonomies often exhibit complex multi-path
hierarchical relationships that existing methods fail to capture adequately, we present ierarchical
and Explicit Label Modeling (HELM) – the first semi-supervised image-based
method capable of modeling multi-path hierarchies in hierarchical multi-label classification.
The framework integrates hierarchy-specific class tokens with graph-based structure
encoders to capture complex relationships among labels, incorporating BYOL as a selfsupervised
learning (Self-SL) module to effectively leverage unlabeled data while maintaining
hierarchical consistency. Through evaluation across remote sensing and medical
imaging domains, we demonstrate that incorporating hierarchical information significantly
improves predictive performance in both supervised and semi-supervised settings.
Through comprehensive evaluation across multiple datasets, we demonstrate that the
proposed methods consistently outperform state-of-the-art supervised and semi-supervised
approaches while requiring significantly less labeled data. The developed frameworks address
fundamental challenges in leveraging unlabeled data for structured prediction tasks
in remote sensing domain, offering practical solutions for large-scale monitoring applications
where annotation costs are prohibitive. The work establishes new benchmarks and
resources for the research community while opening promising directions for future investigation
in automatic hierarchy discovery, visual-semantic alignment, and multi-modal
learning scenarios.