REPOSITORY > RESULTS

Abstract

In this thesis, we develop feature ranking methods for a variety of learning settings, bridging
the gap between the ever more complex data on the one hand, and the lack of feature
ranking methods that would explain the models learned on these data on the other hand.
The developed feature ranking methods address complex machine learning tasks from
supervised, semi-supervised and unsupervised learning: supervised and semi-supervised
structured output prediction (SOP), including multi-target regression (MTR), multi-label
classification (MLC), and hierarchical multi-label classification (HMLC). We also extend
feature ranking methods to relational learning, where the data representation is even richer
and more complex. The feature rankings produced by our methods offer state-of-theart
performance in all the learning contexts, as showcased by extensive empirical studies
presented in this thesis.
The developed feature ranking methods handle various learning contexts in an elegant
and unified way. More precisely, we adapt two groups of feature ranking methods. The
first group of feature ranking methods is tree-ensemble-based and contains the Symbolic,
Genie3 and Random Forest scores. The scores are computed from different ensembles
of predictive clustering trees (PCTs), i.e., random forests, bagging, and ensembles of extremely
randomized PCTs. The second group of methods are distance-based methods that
follow the Relief approach to feature ranking.
In the thesis, we first give the necessary background and present the main ideas behind
the developed feature ranking methods. Then, extensive empirical studies are presented,
evaluating the developed methods for each learning setting. The main empirical findings
are that the proposed feature ranking methods yield meaningful rankings and outperform
the existing methodology. Especially when the number of features is large, ensemble-based
methods outperform the distance-based ones. Both groups of methods scale-up well, since
they are subquadratic in the number of features and easily parallelizable.
The thesis also presents a practically relevant case study that uses the developed
ensemble-based feature ranking algorithms for MTR to explain the models that predict
the thermal power consumption of Mars Express spacecraft of the European Space Agency
(ESA). We finish the thesis with a comprehensive discussion of its contributions and an
outline of many possible directions for further work.