Doctoral dissertation
Knowledge Acquisition through Natural Language Conversation and Crowdsourcing
Views: 4 | Downloads: 0
Abstract
Knowledge Acquisition is an important part of Artificial Intelligence. Having high-quality
structured knowledge helps to advance other research activities that depend on a rich
understanding of the world around us. Due to a lack of reliable automation strategies this
problem has been mostly attacked through hand-annotation and structuring by human
experts. In recent years, there have been attempts to scale the manual approach through
crowd-sourcing.
Although numerous systems and methods for crowd-sourced knowledge acquisition have
been developed to solve the problem of manpower, the issues of high cost, task specification,
targeting, consistency, and eventual quality of acquired knowledge, seem to persist.
The thesis addresses these issues by formalising and implementing an approach to scalable
human-driven knowledge acquisition that significantly reduces the costs involved, and
increases the quality of acquired knowledge. Through a novel set of tools, natural language
user interaction, and a rich background knowledge base, our approach uses contextual clues
to pick the right users at the right time: leveraging existing knowledge to check the consistency
of user-provided answers. Newly acquired knowledge is incorporated into the
knowledge base, creating a virtuous cycle whereby the knowledge acquisition process and
the acquired knowledge are constantly improving through time. The proposed approach
can be incorporated into natural language Human-Computer Interaction (HCI) interfaces
to add high quality, cost-effective knowledge acquisition capabilities to almost any system
with a distributed user base.
We propose a concrete implementation of the system, which has been tested over multiple
years with thousands of users, yielding very promising results. During the test run, we
collected over 57,978 answers, resulting in 386,980 new assertions. Evaluation shows the
extracted knowledge to be true and useful 95% of the time. Furthermore, using context
to pro-actively drive knowledge acquisition increased engagement and effectiveness (the
number of new assertions/day/user) by 175% over the baseline.