REPOSITORY > RESULTS

Abstract

In this thesis, we are dealing with the task of semi-supervised learning (SSL) for structured
output prediction (SOP). In contrast to traditional supervised machine learning methods,
which use only labeled data, semi-supervised methods additionally use unlabeled data. Due
to the laborious annotation procedure, labeled data are a limited asset in many real-life
problems, which can hinder the predictive performance of supervised algorithms. Unlabeled
data, on the other hand, are often much easier to obtain. SSL aims to exploit unlabeled
data to achieve better performance than can be achieved by labeled data alone.
Structured output prediction is concerned with predicting structured, rather than scalar
values, such as multiple classes/variables, hierarchies or sequences. Such (i.e., structured)
outputs are encountered in many applications of predictive modeling. Compared to SSL for
primitive outputs, SSL for SOP received much less attention in the scientific community,
although the need for SSL is even stronger there: Obtaining labels of structured data is
even harder. Furthermore, this field lacks interpretable methods and methods that can
handle various SOP task.
To overcome these issues, we extend the predictive clustering (PC) framework towards
SSL. The PC framework is implemented using predictive clustering trees (PCTs) which can
efficiently handle various SOP tasks. In this thesis, we contribute to the field of SSL for SOP
with two classes of methods stemming from the PC framework. The proposed methods
can handle the following SOP tasks: multi-target regression, multi-label classification and
hierarchical multi-label classification.
The first class of methods is based on the self-training paradigm if it uses its own most
reliable predictions in the learning process. The second class of methods is based on the
extension of the variance functions of PCTs in order for it to accommodate both labeled
and unlabeled examples. We extensively evaluated the proposed methods on a wide variety
of datasets from various domains and with different types of structured output.
The self-training approach offers a state-of-the-art predictive performance on multitarget
regression problems, while producing black-box models and with the cost of increased
computational complexity (due to iterative training of the base model) as compared to
supervised random forests. Semi-supervised predictive clustering trees, on the other hand,
produce readily interpretable models, which are often considerably more accurate than the
corresponding supervised models for structured outputs. The semi-supervised predictive
clustering trees (and ensembles thereof) also exhibit attractive predictive performance on
machine learning tasks with primitive outputs.
Finally, we conduct two case studies where we demonstrate the practical usability of
the proposed methods. In the first case study, we apply the self-training method for
multi-target regression to predict the properties of potential antimalarial medications and
show that the semi-supervised model is more accurate than the supervised model. In the
second case study, we demonstrate on the problem of water quality prediction that semisupervised
PCTs can efficiently solve a practically relevant machine learning task: learning
from partially labeled data.