REPOSITORY > RESULTS

Abstract

Feature construction, encompassing both feature engineering, which involves the manual
design of features by domain experts, and representation learning, which refers to the automated
discovery of useful data representations during model construction, is a fundamental
aspect of machine learning. Its goal is to transform raw data into a more suitable form
that can be exploited by machine learning approaches. While feature construction is wellexplored
in domains like natural language processing and computer vision, where it has
been used in many applications, it has not been explored to the same extent in time-series
analysis and remains largely unexplored in single-objective optimization.
In the domain of single-objective continuous optimization, only a handful of approaches
exist. The primary goal of feature construction in this context is to generate features that
capture properties of the objective function, such as the number of optima or the difficulty
of finding them. Another important application of feature construction in optimization is
algorithm selection, where the properties of the objective function are used to recommend
the most suitable optimization algorithm for a given optimization function. While existing
approaches in this area are capable of capturing some underlying properties of objective
functions, they fall short, particularly in the task of recommending the most suitable
optimization algorithm for a given objective function. This thesis proposes two new feature
construction approaches. The first approach utilizes topological data analysis to extract
features from samples drawn from a given objective function. This method captures various
topological rotation and translation invariant properties that serve as descriptors of the
objective function. The second proposed feature construction technique also uses samples
drawn from the objective function, but applies random transformations to obtain a highdimensional
representation of the objective function. Both approaches are evaluated across
multiple tasks, including differentiating between objective functions, predicting high-level
properties, and algorithm selection. While these approaches demonstrate effectiveness
for differentiating between problems and high-level property prediction, some limitations
remain, particularly in algorithm selection.
The second part of the thesis focuses on feature construction techniques for time series
data. We explore two tasks. The first is investigating the importance of time series features
when used for selecting the most suitable forecasting algorithm. In this context, there are a
handful of research approaches that have explored algorithm selection, but unfortunately,
most of the proposed techniques only show that such techniques can be used for algorithm
selection but do not provide explainability on how and why the most suitable algorithms
are selected. We show that an explainable algorithm selection pipeline can be constructed,
which provides consistent explainable decisions across different pipeline configurations. In
the second task, we address the robustness of constructed representations for the task of
classification. Time series classification has been largely explored, but there are still some
challenges, such as how robust the constructed features are to various distortions and how
this affects the accuracy of downstream classifiers. We explore the topic of robustness and
provide some explanations as to why certain representations are more robust than others.