Review Comment:
Summary:This paper proposed a benchmark that optimizes SPARQL queries on foundational based domain ontologies. The authors employed the benchmark for evaluating the performance of different triple stores. A foundational indexing technique was also designed to achieve faster results.
While this paper is quite easy to follow, it is not innovative enough and has some flaws in experiments to be accepted this time. I'll elaborate on these problems as follows.
Major concerns:
1). The motivation is not clear:
a) Page 6, “relation to our approach” part: Why not design unified foundation Ontology (UFO) for the existing LUBM and UOBM.
b) There is no doubt that using the index in the database can speed up the efficiency of query and access. Why design a special UFO index? Please give more explanations.
c) As for the inconsistency problem of queries you mentioned, I did not find any corresponding solutions in the paper.
2). Experiments are not well-designed and reported:
a) The data source is unknown, just know that it is generated from existing real-world data. But I cannot find any other information about the data, such as the scale of RDF triples and the expressivity of upper ontology.
b) The 16 patterns in Table 3 do not seem representative. In terms of query types, these patterns do not cover all query types, and they are all simple questions. There are no patterns and comparison experiments on complex questions.
c) The comparison of experiments is not fair. As I said before, why not directly use LUBM and UOBM to design UFOs for comparison?
3). Writing issues: I think the authors could do with more polish:
a) The authors mentioned OWL DL, however, as throughout the text I found the data in this paper was only related to the storage format of RDF. The author did not introduce the scale and expressivity of ontology in the experiment.
b) Both the access links of Aviation Safety Ontology and UFO-based Data Generator have been invalidated, resulting in a problem for verifying the method.
c) At the end of Introduction, the author said “This benchmark can be reused not only for our foundational generated data but also for all data sets compliant with the unified foundational ontology”, however, I only found that the event-related ontology design problem is solved, not all UFO compatible data sets
d) Why describing all the ontologies by UFO diagrams instead of direct axioms? Moreover, the ontology cases given in the paper are too simple. Personally, I do not think that UFO language can equivalently describe the ontology under OWL DL.
4). Important references are missing:
a) The original sparql1.1 references are not cited, see [1] below.
b) OBDA and SPARQL optimization work are not cited, such as [2-4]
d) It is suggested that related work should be introduced after the Introduction, so that readers can understand the core motivation.
Minor issues:
1. A lot of axioms have format problems, such as (\ exists P.C)
2. The Figures are not clear, the resolution is too low, such as Fig. 3, Fig 10.
3. There are many typos, such as: wil-> will, representation-> representations, table 6.1-> Table 6.1
4. The format of citation is not appropriate, such as [43], [44], [45]-> [43-45].
5. The numbers of Q1-Q3 appear repeatedly in the experiment, it is recommended to distinguish them.
6. The vertical axis of Fig. 4- Fig. 9 and Fig. 11- Fig. 16 lacks a specific time unit.
|