A RADAR for Information Reconciliation in Question Answering Systems over Linked Data

Tracking #: 1365-2577

Authors: 
Elena Cabrio
Serena Villata
Alessio Palmero Aprosio

Responsible editor: 
Guest Editors Question Answering Linked Data

Submission type: 
Full Paper
Abstract: 
In the latest years, more and more structured data is published on the Web and the need to support typical Web users to access this body of information has become of crucial importance. Question Answering systems over Linked Data try to address this need by allowing users to query Linked Data using natural language. These systems may query at the same time different heterogenous interlinked datasets, that may provide different results for the same query. The obtained results can be related by a wide range of heterogenous relations, e.g., one can be the specification of the other, an acronym of the other, etc. In other cases, such results can contain an inconsistent set of information about the same topic. A well known example of such heterogenous interlinked datasets are language-specific DBpedia chapters, where the same information may be reported in different languages. Given the growing importance of multilingualism in the Semantic Web community, and in Question Answering over Linked Data in particular, we choose to apply information reconciliation to this scenario. In this paper, we address the issue of reconciling information obtained by querying the SPARQL endpoints of language-specific DBpedia chapters. Starting from a categorization of the possible relations among the resulting instances, we provide a framework to: (i) classify such relations, (ii) reconcile information using argumentation theory, (iii) rank the alternative results depending on the confidence of the source in case of inconsistencies, and (iv) explain the reasons underlying the proposed ranking. We release the resource obtained applying our framework to a set of language-specific DBpedia chapters, and we integrate such framework in the Question Answering system QAKiS, that exploits such chapters as RDF datasets to be queried using a natural language interface.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Minor Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 02/May/2016
Suggestion:
Accept
Review Comment:

I find the authors revisions satisfactory for publication at this special issue.

There are two remaining weaknesses that I leave it up to the authors to address them in the final version:
First, the evaluation does not clearly show the overall performance (and justify the need for the proposed bipolar argumentation method) within an end-to-end question answering system. You have two examples from QAKiS in Figures 3 & 4. One can be answered by a simple majority voting (Fig. 3) and the other can be a simple grouping based on surface forms. You need better stronger examples.
Second, the online QAKiS demo still does not work. It did not work at the time of the original review, the first revision review, and now for this second revision. The "Reconciliation" tab does not show anything so your screenshots are either no longer working or are done on a different and non up-to-date browser. The SPARQL queries are all in English (the bug is not fixed yet). I also tried a few random questions and none worked well. The system really needs work to become an industrial strength high-quality and mature software system, but it can do better as a research prototype by providing a bug-free smooth experience at the very least.