Exploratory querying of SPARQL endpoints in space and time

Tracking #: 1078-2290

Authors: 
Simon Scheider
Auriol Degbelo
Rob Lemmens
Corne van Elzakker
Peter Zimmerhof
Nemanja Kostic
Jim Jones
Gautam Banhatti

Responsible editor: 
Guest editors linked data visualization

Submission type: 
Tool/System Report
Abstract: 
The linked data Web provides a simple and flexible way of accessing information resources in a self-descriptive format. This offers a realistic chance of perforating existing data silos. However, in order to do so, space, time and other semantic concepts need to function as dimensions for effectively exploring, querying and filtering contents. While triple stores, SPARQL endpoints, and RDF were designed for machine access, large burdens are still placed on a user to simultaneously explore and query the contents of a given endpoint according to these dimensions. First, one has to know the semantic concepts and the type of knowledge contained in an endpoint a-priori in order to query content effectively. Second, one has to be able to write and understand SPARQL and RDF. And third, one has to understand complex data type literals for space and time. In this article, we propose a way to deal with these challenges by interactive visual query construction, i.e., by letting query results feedback into both (space-time) exploration and filtering, and thus enabling exploratory querying. We propose design principles for SPEX (Spatio-temporal content explorer), a tool which helps people unfamiliar with the content of SPARQL endpoints or their syntax to explore the latter in space and time. In a preliminary user study on a repository of historical maps, we found that our feedback principles were effective, however, that successful question answering still requires improvements regarding space-time filtering, vocabulary explanation and the linking of space-time windows with other displays.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Major Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Heiko Paulheim submitted on 05/Jun/2015
Suggestion:
Minor Revision
Review Comment:

The paper has been reorganized and extended in various respects.

The system presented is interesting, and it is useful at least in a particular (however, not very mainstream) use case. The evaluation still does not contain any comparison to other systems and/or baselines.

While the paper would make a nice conference paper, e.g., at conferences such as ISWC, my feeling is that it is not yet mature enough for a journal publication. The authors themselves state that their "study was preliminary" and "does not yet allow drawing representative conclusions" - this is not the degree of maturity expected for a journal publication.

Some ideas for reshaping the paper: the authors could make some statements about the usage of space and time related vocabularies in the LOD cloud as to provide an argumentation for a wider range of possible use cases. Furthermore, if an evaluation against other tools seems complicated or inappropriate, they could at least compare SPEX with and without timeline and map features activated.

Further remarks:

On page 2, the authors claim that faceted browsing does not allow users to perform complex searches. I don't think this is true: it allows to subsequently refine a search. In the use case, the users could first search for a map by a keyword and then refine by adding facets restricting the time and geographic area, as well as topical facets. I furthermore don't agree that (p.4) faceted browsing always requires a local focus, it rather restricts a *set* of objects along different dimensions.

On page 16, the authors state that prefixes of predicates confuse the users. Actually, labels should be used for presentations, not URIs.

The analysis in 6.3 is a bit superficial. I would have expected some statements about the percentages of eye gaze at the map, timeline, query and results widgets.

Tables 2 and 3 should be merged. It is furthermore not clear why user 0 is in table 3, but not in table 2

Minor issues:
* 4.1 should rather be a first level section, or the part of section 4 preceding 4.1 should be 4.1, with 4.1 becoming 4.2
* section 5 should make a statement about the size of the dataset used in the xperiments

Review #2
Anonymous submitted on 23/Jun/2015
Suggestion:
Accept
Review Comment:

The authors revised and extended their paper significantly. They have added contents in nearly all parts of the paper. The related work is now better integrated and the evaluation has been complemented by eye tracking results. The authors clarified many of the issues raised by the reviewers. Among others, they made clear that the presented SPEX tool is currently only an interactive prototype and not a mature system (which is also my impression from trying it). This makes me wondering whether the paper really fits into the category "Tools and Systems Report" (though I lean towards answering this question with "yes"). Further, the evaluation is only "preliminary", while readers of a journal article usually expect a substantial evaluation of the presented work. However, I still believe the work fits well within the scope of this special issue and would therefore recommend acceptance of the article.

I am glad to see that the authors addressed most of my comments. Unfortunately, I could not find any answer to the following questions from my first review: "Since the participants of the user study needed at least 45 minutes to familiarize themselves with the tool, one could question if the tool is really appropriate for lay users? Can it be expected that lay users are willing to learn a tool like this for nearly an hour in non-controlled settings?" and "The tasks of the scenario for the user study all seem to be of a similar type and to follow a similar pattern. Could this have had an effect on the user study? Is a larger variation in task types a problem for the approach and/or developed prototype?" While I would have expected that the authors provide answers to these questions in the article or at least in the cover letter, I do not consider that critical for the final decision on acceptance or rejection of the article. However, I would appreciate if the authors could provide answers together with the next revision of their article.

From my point of view, the paper is nearly ready for publication. However, for the camera-ready version, the authors should make sure that all mentioned related work is referenced. For instance, there are currently no references for the mentioned tools RelFinder, SPARQLViz, LDVizWiz, and CODE wizard. The authors should also carefully check the capitalization in the reference list. Acronyms like SPARQL, DBMS, or LESS are currently written in small letters in the listed paper titles, and tool names are also wrongly capitalized (see references of gFacet, RDF-GL, or ViziQuer). In addition, the authors may incorporate recent developments in the final version, such as LodLive or QueryVOWL, that are both quite related (LodLive even features a geographic map). Finally, they should check the paper for typos (e.g., wrong use of articles) and could further improve the formatting (e.g., pagebreaks on pages 4, 9, and 11, format of reference [36]). I trust the authors that they will carefully address these issues in the final revision, and therefore vote for "accepting" the paper without insisting on a check of the final revision.

Review #3
By Mariano Rico submitted on 02/Jul/2015
Suggestion:
Accept
Review Comment:

This manuscript was submitted as 'Tools and Systems Report' and should be reviewed along the following dimensions: (1) Quality, importance, and impact of the described tool or system (convincing evidence must be provided). (2) Clarity, illustration, and readability of the describing paper, which shall convey to the reader both the capabilities and the limitations of the tool.

I see a good evolution in the paper. Most reported issues were managed efficiently.
Some minor typos:
- Figure 4 is before figure 3. There is a LaTeX command to solve it.
- If you want to show all maps from a given time period, you have to create a variable Map, close the window, click again in the variable, and now you can filter. Why do you have these two windows instead of a unique window?.
- page 2, footnote 6, the URL provides a 503 error.

Although the evaluation uses a small number of users and, therefore it is not possible to get statistic significance measures, the detailed discussion on the cases is good enough for me.

I would like to know why users have severe problems to solve complex queries. You provide a general discussion in section 6.4 but it does not cover all my questions.


Comments

Dear editors,

Please find attached a largely revised version of our paper. In summary, we did the following:
- Re-organized the whole paper such that it better follows our argumentation structure: motivation, related work, design principles, SPEX with functional comparison, scenario description, evaluation, conclusion
- Strengthened the motivation, re-focused the paper on the role of space-time interfaces in exploratory querying, added corresponding research questions, clarified the contribution of the paper with respect to earlier work.
- Improved section 2 (related work): refocused on space-time, shortened general linked data aspects and added a discussion of a variety of related tools that we missed in the last version. We also discussed particular challenges regarding the integration of space-time interaction into exploratory querying.
- Section 3: reorganized design principles
- Section 4: Better described the tool and its scope (prototype), improved figures to illustrate what can be done with SPEX, added a detailed explanation of the flow chart (figure 6) and a step-by-step illustration of a spex query with screenshots (Figure 7). Furthermore, added gFacet to the functional tool comparison and argued in how far SPEX is functionally different
- Section 5: better motivated the scenario questions and explained the expected answers in detail, added a new Figure (8) to this end.
- Section 6 (evaluation): 1. better explained the scope and purpose of the user test, 2. discussed general results and 3. added a new section about the role of space-time interfaces in exploratory querying. We investigated the usage of the space-time window across users and questions (table 3) and analysed corresponding gaze plots. Main insights of the user study are summarized at the end.

Thanks for your detailed feedback,
Simon Scheider