The Semantic Web: Two Decades On

Tracking #: 2229-3442

Authors: 
Aidan Hogan

Responsible editor: 
Guest Editor 10-years SWJ

Submission type: 
Other
Abstract: 
More than two decades have passed since the establishment of the initial cornerstones of the Semantic Web. Since its inception, opinions have remained divided regarding the past, present and potential future impact of the Semantic Web. In this paper -- and in light of the results of over two decades of development, not only on the Semantic Web, but also related technologies -- we reflect on the current status of the Semantic Web: its successes, its failures, its challenges, its opportunities and its potential impact on the future Web. We being by playing devil's advocate to the original vision of the Semantic Web, reviewing some of the external criticism of this vision that has been put forward by various authors; we draw together the individual critiques, arguing both for and against each point based on the current state of adoption. We then present the results of a questionnaire that we have posed to the Semantic Web community in order to understand its perspective(s) regarding the degree to which the original Semantic Web vision has been realised, the impact it can potentially have on the Web (and other settings), its success stories thus far, as well as the degree to which the community agrees with the aforementioned critiques of the Semantic Web in terms of both its current state and future feasibility. We conclude by reflecting on both the successes and failures of the Semantic Web initiative thus far, we well as future challenges and opportunities.
Full PDF Version: 
Tags: 
Reviewed

Decision/Status: 
Major Revision

Solicited Reviews:
Click to Expand/Collapse
Review #1
By Jérôme Euzenat submitted on 28/Jun/2019
Suggestion:
Minor Revision
Review Comment:

The paper is made of two parts: first an analysis of a set of criticisms made to the semantic web discussing arguments for and against the criticism. Then the paper discusses the results of a pool made among semantic web stakeholders about these issues.

I read the paper in its initial form (without the pool) and then latter in the full form.
I tend to prefer the initial form.

The paper is interesting in its discussion of the objections to the semantic web and fits perfectly the topic of the special issue.
It is very well-written.

An issue I have is that the criticisms do not apply at the same level: some are rather marketing (what is the business model, niche), social (depends on agreement, on reliability), economical (cost/incentive), technical/rant (too verbose) and even sometimes phrased as an opinion/prospective (will not scale, will be made redundant).
I would have liked to see this acknowledged, for instance by organising the presentation with respect to different types of criticisms or, for the latter, by phrasing them in a more neutral way. It would also have been worth to look at the opinion underlying some criticism: what is wrong with solving niche problems (rare illnesses are also niche markets and who think that the semantic web's aim is to solve problems)? There cannot be reliable publishers...

Another point is that these kinds of criticisms come from different origins, that it would have been worth identifying as well (niche problem and verbose standards do not come from the same audience and will not be addressed by the same actor).
However, in spite of these remarks of organisation, I found that part of the paper clear and pleasant to read, as well as well-balanced in the arguments (I do not think that the author was really the Devil's advocate, but I may be biased).

I found the second part less convincing. I would explain it in the following way:
The criticisms of the first part emanated from likely one precise individual and I can understand the poor programmer complaining about the lack of tools or the blogger-futurologist announcing the end of the semantic web. They stand from different positions.

In the pool, it is unclear to me what the group of respondent represents and what its interest to comment on the criticisms is. Actually, this group is certainly best characterised by its answers to the pool.

I also agree with another commentator that the questionnaire is too tied to technologies/standards, though the criticisms came from many different perspectives.

A few point of disagreements:
Note that there seems to be some confusion to what means for a standard to be verbose: the size of the standard document? Then XML has no reason to qualify. The size of what is expressed in that standard? Which highly depends on the format.

How comes that the semantic web has been designed from scratch? It reuses the basic infrastructure of the web... For going on the ontology side, it reuses description logics which were quite clearly defined at the moment they were adopted (and do not need many pages to be defined either). Can't this be seen as an evolution?

Details:
- abstract: 'We being by playing', I assume that this is 'begin'
- p4, last paragraph of 2.3, isn't it 'users should _not_ blindly trust results'?

Review #2
By Frank van Harmelen submitted on 15/Jul/2019
Suggestion:
Reject
Review Comment:

The manuscript consists of two parts. In the first part it summarises a series of critiques on the semantic web programme in terms of attacking and defending arguments, the second part presents a survey among semantic web researchers and practicioners on each of these arguments.

I have a number of objections to this paper which are not easy to
repair, hence my choice for a reject instead of a major revision:

Concerning part 1, I question whether it is useful to speak about the
research of the past 20 years in terms of "success" and "failure", as if it were an R&D programme.

Concerning part 2, I doubt the validy of any insights from the survey
because of the small number of respondents and the skewed nature of
the participants.

Part 1:

The entire paper is framed in terms of "succes" or "failure" the a
research programme. I question whether this framing is useful or even
meaningful. The author treats the activities of the semantic web
community over the past 20 years as a R&D programme with a specific
target, for which it would be meaningful to discuss whether the
targets have been met. I strongly disagree with this framing. No 20
year research programme by an international community of thousands of
students and researchers can be measured against targets as if it were
an R&D programme executed by a single organisation. (I realise that
the funding organisations for which we have to write our project
proposals force us into talking about deliverables, but this is not
how progress in science should be measured).

Firstly: even from the very early days there were debates what the "goals" of
the Semantic Web should be (if any): should it be about enriching the WWW of
text and pictures? Should it be about publishing data on the Web
(which is an altogether different activity than enriching the WWW of
text and pictures). Should it be about data interoperability (even if
that data is not on the web at all)? Speaking about "the" success or
failure implicitly assumes that there was such a coherent target
goal. There isn't, and there wasn't. And that's not necessarily a
problem. People working with these different motivations in mind can
fruitfully collaborate on topics where they share an interest, even
though their "success" or "failure" targets (if any) are entirely
different.

Secondly: even when there would be a (nebulous) "target", then
progress in science can barely be measured by whether or not a field
achieved it's original target. Take AI as an example: there is a very
nebulous target, and after 60 years, the field is not even close to
achieving anything like it. So does that mean that AI has "failed"?
Clearly not, because on the way, AI has obtained a large number of
fundamental insights into whole families of representations and
algorithms, and has produced all kinds of useful spin-offs. The
Semantic Web is in precisely that situation: it has obtained a large
number of fundamental insights into whole families of representations
and algorithms, and has produced all kinds of useful spin-offs. With
this in mind, it doesn't make much sense to speak about whether the
semantic web has "succeeded or failed".

(as a final comment: I'm not sure if it is the job of a paper in
scientific journal to respond to every blogger with an opinion, but
I'll not go into that).

Part 2:

Although I appreciate the intention of the authors and the effort he
took, I think the questionaire (and with it its results) are
fundamentally flawed, for two reasons: First: at 113, the number of
participants is actually rather limited. Secondly (and more
importantly), almost 50% of the respondents either academic or partly
academic.

This means that a number of the responses have a very high degree of
"they would say that, wouldn't they". Of course the respondents would
say they are optimistic about future chances for the semantic web
enterprise (otherwise: why would they be in it at all?). Similarly,
the wordcloud of "success stories" mirrors exactly what an outsider
might call the echo chamber in the semantic web circuit.

Summarising:

(1) I think the paper misconstrues the past 20 years of semantic web
research as an R&D programme with a coherent target about which it would
be meaningful to speak in terms of success or failure to attain its
targets, and
(2) I think the questionaire is flawed in terms of both size (small)
and background (too uniform) of the participants.

Becaus neither of these points can IMHO be fixed without
deconstructing the entire thinking behind the paper, I must suggest
the paper be rejected.

Review #3
By Jens Lehmann submitted on 30/Aug/2019
Suggestion:
Minor Revision
Review Comment:

The article describes opinion on the success and failure of two decades of developments of semantic web technologies.

After a brief overview, the article gives an overview of some of the recent major critiques of the semantic web. For each critique, arguments for and against it are given. All arguments are understandable from my point of view and it is good to see a carefully crafted collection of those. However, what I would prefer even more is a systematic way in which the general criticisms are obtained. It could be that the 10 listed criticisms are not fully representative of opinions given by (potential) users of semantic web technologies.

A further potential issue is that the paper does not actually take a position (apart from a comment later in Section 4). In this sense, it is not a typical position paper - it rather collects different points of view rather than arguing for an own point of view or propose future directions. From my point of view, for this particular special issue, this is fine and does not hinder acceptance of the paper.

In the subsequent section, the paper describes the results of questionnaire, which was filled by 113 users after an announcement on the Semantic Web mailing list. While the results are not entirely surprising to me, the analysis is well done and the number of responses makes it a valuable mirror of opinions of the semantic web community on their own technologies and future prospects.

Overall, I believe the article is useful for the community and recommend the following minor revisions before acceptance:

- An explanation how the 10 criticism were collected.
- "The Semantic Web addresses a niche problem" => The answers seem to mostly focus on search on the web, but this is only one application area.
- In Section 2, I would welcome more statistical evidence in the arguments for and against a particular criticism (if available).
- An extension of the forward looking discussion in Section 4.

Other comments:

- 78 users with strong expertise in the questionnaire means that there is a risk of collecting mostly information from inside the core communinity rather than the wider community.
- "users should blindly trust results" => "should not"