Review Comment:
The manuscript consists of two parts. In the first part it summarises a series of critiques on the semantic web programme in terms of attacking and defending arguments, the second part presents a survey among semantic web researchers and practicioners on each of these arguments.
I have a number of objections to this paper which are not easy to
repair, hence my choice for a reject instead of a major revision:
Concerning part 1, I question whether it is useful to speak about the
research of the past 20 years in terms of "success" and "failure", as if it were an R&D programme.
Concerning part 2, I doubt the validy of any insights from the survey
because of the small number of respondents and the skewed nature of
the participants.
Part 1:
The entire paper is framed in terms of "succes" or "failure" the a
research programme. I question whether this framing is useful or even
meaningful. The author treats the activities of the semantic web
community over the past 20 years as a R&D programme with a specific
target, for which it would be meaningful to discuss whether the
targets have been met. I strongly disagree with this framing. No 20
year research programme by an international community of thousands of
students and researchers can be measured against targets as if it were
an R&D programme executed by a single organisation. (I realise that
the funding organisations for which we have to write our project
proposals force us into talking about deliverables, but this is not
how progress in science should be measured).
Firstly: even from the very early days there were debates what the "goals" of
the Semantic Web should be (if any): should it be about enriching the WWW of
text and pictures? Should it be about publishing data on the Web
(which is an altogether different activity than enriching the WWW of
text and pictures). Should it be about data interoperability (even if
that data is not on the web at all)? Speaking about "the" success or
failure implicitly assumes that there was such a coherent target
goal. There isn't, and there wasn't. And that's not necessarily a
problem. People working with these different motivations in mind can
fruitfully collaborate on topics where they share an interest, even
though their "success" or "failure" targets (if any) are entirely
different.
Secondly: even when there would be a (nebulous) "target", then
progress in science can barely be measured by whether or not a field
achieved it's original target. Take AI as an example: there is a very
nebulous target, and after 60 years, the field is not even close to
achieving anything like it. So does that mean that AI has "failed"?
Clearly not, because on the way, AI has obtained a large number of
fundamental insights into whole families of representations and
algorithms, and has produced all kinds of useful spin-offs. The
Semantic Web is in precisely that situation: it has obtained a large
number of fundamental insights into whole families of representations
and algorithms, and has produced all kinds of useful spin-offs. With
this in mind, it doesn't make much sense to speak about whether the
semantic web has "succeeded or failed".
(as a final comment: I'm not sure if it is the job of a paper in
scientific journal to respond to every blogger with an opinion, but
I'll not go into that).
Part 2:
Although I appreciate the intention of the authors and the effort he
took, I think the questionaire (and with it its results) are
fundamentally flawed, for two reasons: First: at 113, the number of
participants is actually rather limited. Secondly (and more
importantly), almost 50% of the respondents either academic or partly
academic.
This means that a number of the responses have a very high degree of
"they would say that, wouldn't they". Of course the respondents would
say they are optimistic about future chances for the semantic web
enterprise (otherwise: why would they be in it at all?). Similarly,
the wordcloud of "success stories" mirrors exactly what an outsider
might call the echo chamber in the semantic web circuit.
Summarising:
(1) I think the paper misconstrues the past 20 years of semantic web
research as an R&D programme with a coherent target about which it would
be meaningful to speak in terms of success or failure to attain its
targets, and
(2) I think the questionaire is flawed in terms of both size (small)
and background (too uniform) of the participants.
Becaus neither of these points can IMHO be fixed without
deconstructing the entire thinking behind the paper, I must suggest
the paper be rejected.
|