REPOSITORY > RESULTS

Abstract

With the resurgence of neural network-based learning in the last decade, machine learning
methods are becoming critical components of many real-life intelligent systems. However,
while being able to learn effectively and at scale, such systems are often non-interpretable
and unable to exploit existing symbolic background knowledge. The paradigm that offers
such endeavour is symbolic learning, which has been investigated for more than 50 years.
The main focus of this thesis is the recent paradigm of neuro-symbolic machine learning.
This branch of learning investigates whether combining the techniques from neural
(sub-symbolic) and symbolic machine learning can be used to develop better performing
and more explainable predictive models. The notion of neuro-symbolic machine learning
transcends individual input data types and can be considered when learning from
tables, graphs, and text-based data. This thesis aims to investigate when, and to what
extent, can the neuro-symbolic paradigm prove beneficial when learning network node
representations, classifying texts and ranking features. Furthermore, we investigate different
aspects of neuro-symbolic models: from predictive performance to scalability. The
contributions of this thesis address different input data types. We first present the results
of applying neuro-symbolic learning to relational learning, considering two different
relational learning scenarios: learning from relational databases and network node embedding.
We demonstrate that by using the neuro-symbolic paradigm, improved scalability of
propositionalization-based approaches is achieved. Further, by considering neuro-symbolic
node representation learning, we demonstrate competitive predictive capabilities, while, by
systematic (symbolic) node pruning, reaching better scalability.
Next, we present autoBOT, a neuro-symbolic autoML system for automating text classification.
By simultaneously considering both symbolic and sub-symbolic document representations,
evolution-based optimization yields well-performing models that remain interpretable
at two different granularities: at the level of feature types (i.e., which feature type
is more relevant) and also at the level of individual features. To our knowledge, autoBOT
is one of the first neuro-symbolic autoML systems aimed at optimizing both performance
and explainability. A novel, previously unpublished contribution is also a computational
framework, enabling us to scale autoBOT to a supercomputing grid, allowing two to three
orders of magnitude more experiments to be conducted on a single machine at the same
time.
The final part of the thesis focuses on of feature ranking. This task addresses the issue
of identifying importance scores for features, which correspond to their capacity to separate
parts of the target space – highly ranked features are commonly the ones that impact the
learning process the most. We demonstrate that the neuro-symbolic paradigm helps better
understand the relationship between neural attention and ranking and offers better scaling
of existing feature ranking approaches, such as the Relief-based feature ranking approaches.
The thesis concludes with an evaluation of the developed approaches, lessons learned and
guidelines for potentially interesting future work.