REPOSITORY > RESULTS

Abstract

The proposed dissertation belongs primarily to the field of machine learning on the one
hand, but also to the field of soil science on the other hand. In terms of machine learning, it
is concerned with the improvement of existing machine learning algorithms for predicting
structured outputs, more specifically for multi-target prediction. In terms of soil science, it
addresses two case studies of applying machine learning methods for multi-target prediction
to two practical problems of modeling two different soil functions from data in the context
of Irish agriculture.
The majority of approaches for multi-target prediction (MTP) do not explicitly take
into account the dependencies among the multiple targets. In order to address this draw-
back, in the proposed dissertation, we propose approaches that find dependencies in the
target space by explicitly structuring, in a hierarchical manner, the different targets. Using
different representations of the target's attributes (based on the feature importance scores
of the input attributes for predicting each target), we use hierarchical clustering of the tar-
gets. Having discovered a hierarchy on the target space, we obtain a reformulation of the
original task of multi-target prediction into a task of hierarchical multi-target prediction.
We then employ approaches for hierarchical multi-target prediction on the transformed
task, expecting improved predictive performance.
We address two tasks of MTP, namely multi-label classification (MLC) and multi-
target regression (MTR). In both cases, we use feature importance estimation based on
tree-ensembles, for classification and regression, respectively, based on the GENIE3 ap-
proach. We use different clustering approaches for structuring the target space, includ-
ing balanced k-means, agglomerative clustering, and predictive clustering trees (PCTs):
Of these, balanced k-means gives the best results. On the hierarchical versions of the
problems, we use PCT ensembles for hierarchical MLC (HMLC) and hierarchical MTR
(HMTR), respectively.
We conduct an extensive experimental evaluation on various benchmark datasets for
MTP (MLC and MTR) tasks, showing the advantage of using our proposed method for
structuring the output space. Using ensembles of PCTs for HMLC and HMTR on the
structured output spaces performs clearly better than using PCT ensembles for MLC and
MTR on the original spaces. The differences in performance are largest for large output
spaces (with more than 100 targets).
We also address two case studies of applying machine learning methods for multi-target
prediction to two practical problems of modeling two different soil functions from data in
the context of Irish agriculture. The data were provided by TEAGASC, Environment Soils
and Land-use Department, from Ireland. TEAGASC was also the source of expertise about
the tasks.
First, we apply PCTs for MTR, as well as ensembles (random forests) thereof to the
task of estimating the total herbage production and nutrient uptake, i.e., the task of mod-
eling the soil function of primary productivity, on Irish dairy farms. We then apply PCTs
(and ensembles) for semi-supervised MTR to model a combination of another two soil functions, i.e., water regulation and purification, and provision and cycling of nutrients. More specifically, we learn models for assessing the chemical quality (nitrogen and phosphorus
loss from soils through runoff and leaching) and the biological quality of water in Irish
agricultural lands. In the latter case, we used incompletely (partially) labeled data, which
has missing values for the target variables we want to predict. This is an innovative use
of semi-supervised PCTs for MTR, as only fully labeled (all target values present) or fully
unlabeled (no target values) examples had been used so far, whereas the real-world data
from this study has partially labeled examples (with some but not all target values).
In both case studies, models are learnt in the form of PCTs and PCT ensembles. They
are both accurate (especially ensembles) and understandable (individual PCTs). They
reveal knowledge about the studied domains, which is both consistent with existing knowl-
edge of domain experts and provides new insights, important for practical use in the context
of achieving better soil function outcomes for given fields/agricultural lands.