Review Comment:
Overall evaluation
Select your choice from the options below and write its number below.
== 3 strong accept
2 accept
== 1 weak accept
== 0 borderline paper
== -1 weak reject
== -2 reject
== -3 strong reject
Reviewer's confidence
Select your choice from the options below and write its number below.
5 (expert)
== 4 (high)
== 3 (medium)
== 2 (low)
== 1 (none)
Interest to the Knowledge Engineering and Knowledge Management Community
Select your choice from the options below and write its number below.
5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
Novelty
Select your choice from the options below and write its number below.
5 excellent
== 4 good
== 3 fair
== 2 poor
== 1 very poor
Technical quality
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
3 fair
== 2 poor
== 1 very poor
Evaluation
Select your choice from the options below and write its number below.
== 5 excellent
4 good
== 3 fair
== 2 poor
== 1 not present
Clarity and presentation
Select your choice from the options below and write its number below.
== 5 excellent
== 4 good
== 3 fair
2 poor
== 1 very poor
Review
The paper presents an exploratory and experimental approach to assessing the use and effects of the 'properties-based approach' and 'class-based approach' and a hybrid model with respect to adding subclasses (and defining them) and sub-properties, starting with ODPs and their use in the field, then also other ontologies, and closing off with performance. The paper meanders around following the author's journey, but the multiple new insights presented are definitely useful for ontology engineering.
Notwithstanding, there's room for improvement. And it would be a plus if the test data can be made available online.
Detailed comments:
The abstract is too short and does not cover the paper's contents (nor does the title) or the main results. e.g., on p2 there's a paragraph on "main contributions…", some of which can go in the abstract.
Section 1. Introduction
it is irrelevant that WOP is held in conjunction with ISWC, and can be removed
Section 3.
In the study set-up, it has not been made clear why Linked Open Vocabularies and LOD data should be included, as opposed to just ontologies. Later on in the paper the author zooms in on desiring to query and classify instances, but this is not obvious upfront (and wouldn't be the only scenario).
That so little different types of ODPs have been used (table 2) is interesting of itself, and worth looking into.
Section 3.1 already introduces the "property-oriented" and "class-oriented" notions, but they are only properly introduced in section 3.2.
p6 "ODP specialisations that modify the semantics of object properties", and, similar, on p9 first sentence of section 4: no, not w.r.t. the language. Arguably, the subject domain semantics. I'm assuming the author refers to 'modifying' in the sense of changing the domain and/or range declarations, but that should be made clear.
"Formally, the owl:subPropertyOf definition implies … are more narrow than those of their superproperties.". No, that is not what the OWL standard says. It means that when OPE1 isa OPE2, then "if an individual x is connected by OPE1 to an individual y, then x is also connected by OPE2 to y". What the author assumes, and wants it to be, is analyzed and discussed in detail in:
Keet, C.M. Detecting and Revising Flaws in OWL Object Property Expressions. 18th International Conference on Knowledge Engineering and Knowledge Management (EKAW'12), A. ten Teije et al. (Eds.). Oct 8-12, Galway, Ireland. Springer, LNAI 7603, 252-266.
There is also a sloppiness on the logic aspects on p9, first paragraph. Although the second paragraph qualifies the "corresponds" as not to be meant to be equivalence, it goes on later to say in the 'hybrid strategy' section that the class ones are "logically redundant": they are not for the general case. It would be useful to know how often there are equivalences asserted with only one property, but Algorithm 1 does not detect that (it stops at having found one instance, but there may be more. it does not check for the latter, for it's just a single if, and the caption doesn't make it more convincing saying there is a loop: if there was a loop, then it should be a while statement, not an if).
Section 4.
On the strategy effects, "Yet another advantage is that,…": where does that assertion come from? not the author's results (at least, not at that stage in the paper).
"…including in querying, where SPARQL triple patterns…": that just as well can be a disadvantage: with 'underspecified' object properties, one will end up with too many tuples returned by a query, a bunch of them being irrelevant, but which more easily can be avoided with specialization of object properties, and using the specialized property in the query.
Please add the versions of Pellet and HermiT that were used. Also, Konqueror was the best performing reasoner in ORE 2014, showing substantially better performance, which may be worthwhile checking out. Further, I still miss the use case scenario description on why data.
"reported results of the reasoning tasks were equivalent,": I presume the author meant that the deductions--set of inferred axioms--were the same for that test case.
Section 5. Discussion
much leaves to be discussed (and 'limitations', not "delimitations").
While the author seems to be bent on putting everything in an ODP straightjacket, why not, at least, have an ODP for either modeling choice? Or something else?
"If this is the case, and if this more common use of the class-oriented strategy is a consequence": this is just guessing. The ODP-users may well have been influenced by the pizza ontology tutorial that advises against declaring domain and range restrictions, and those other users by conceptual data modeling that introduces new associations/relationship for pretty much everything. whichever.
Typos, presentation:
There are multiple typos, and the paper will need a careful reading to fix them.
Tables should have a horizontal line at the top, not 'open'.
The number of instances of the ways of specialization with the ODPs in section 3.2 ideally shouldn't be buried in the text, as they become important later on. Maybe put them in a table as well, for easy comparison. This I had to do manually myself now with the table and text in first paragraph on p11 and those preceding ODP results paragraphs (but I wouldn't want to have to do that).
Overall, the flow of the paper could be improved upon by, possibly, first, the 'preliminary' experiment that analyses ODP usage (could be shortened), then stating there are these properties-based and class-based approaches, results with a content ODP favoring one and to compare that with 'in the wild' without ODPs, and then trade-offs between the two options including the performance testing. If space is needed: the extreme design easily can be dropped without affecting the flow, results, and message of the paper.
|