How much Knowledge is in Knowledge Graphs? - A Knowledge Management Perspective

I was reading this position paper trying to understand who is the intended audience and what is the takeaway suppose to be.

The paper was submitted to theSWJ so presumably the audience is the semantic web community.

It was not clear what is the takeaway.
- This is suppose to be a position paper but the paper lacks a clear and a specific position statement. Actually, the authors present a question, rather than a position (page 2, line 41) .
- It seems that the takeaway is that the semantic web community should be doing more with the knowledge management community.
- It also seems that the semantic web community has the mechanism, through knowledge graphs, to make a reality a lot of what the knowledge management community has been trying to accomplish.
After reading this multiple times, I'm trying to figure out what is the position.  In my own words, it *seems* that the position is the following:

"knowledge graphs can (and should) be used to create knowledge in three ways by 1) restructuring 2) integrating or 3) enriching, and a new goal is to see if the resulting knowledge graph  can be actionable"

Given that I'm already trying to guess what the position is, this is already grounds for rejection. As a reader, I should not be trying to guess what is the takeaway. Therefore the quality of writing needs .

I noticed that the important aspects of this position paper are presented in the conclusions. This should be presented up front and the rest of the paper is what supports the position.

Furthermore, the survey is weak and reduces the importance of the impact of the position. Reviewing 2 page ISWC industry papers from the past three years is not sufficient to make the wider claims that the authors are trying to make. The authors are preaching to the choir. They must extend and read the vast amount of industry reports and conferences that exist. There is an enormous amount of case studies from vendors, conference talks on youtube .

For example look at all the conference talks from the Knowledge Conference 2019
- Uber: https://www.youtube.com/watch?v=r3yMSl5NB_Q
- Wellsfargo: https://www.youtube.com/watch?v=25UIgiiYqsE
- Knowledge graph and real estate: https://www.youtube.com/watch?v=Bp38pYrpdSY
- ... these are just a couple of ones that appeared when I searched for "knowledge graph"

Also look at the industry talks in the Stanford KG course: https://web.stanford.edu/class/cs520/

This survey doesn't even scratch the surface. And what is the survey suppose to tell me? That we are already on the right path? Or what’s missing?

As a member of the semantic web community who works in academia and in industry, I'm not sure what I'm suppose to take away.

I believe the authors have a valuable intention: the Knowledge Management and Semantic Web community need to work together. However, I don't understand what is the specific call for action. Per the conclusion, there isn't anything novel (i.e. putting humans in the loop isn't a new call for action).

As a position paper this could be just a blog post or an editorial. I can see this being turned into a full survey of KM work that the semantic web community is not aware of and map it to existing contributions from semantic web community and highlight all the open opportunities. This would be valuable.

1) originality: does the paper convey new ideas or perspectives?
It's not clear what is the original idea . I'm making a lot of assumptions.
2) quality of writing
The writing itself is ok (grammar) . But I do not know what the takeaway is, so the writing is not clear.
3) relevance to the community: is the proposed topic relevant to the Semantic Web community
This is a message to the semweb community, but again, I don't know what I'm supposed to take away.
4) potential impact to open up new research directions: in how far are open research questions and directions mentioned and how relevant are these?
Again, not clear what the takeaway is, so I don't know what the potential impact is.
5) interdisciplinarity: in how far does the paper enable/foster interdisciplinary discussions connecting to other communities?
It seems that a takeaway is that the semantic web community should work more with the knowledge management community, but it's not clear exactly wh

This paper offers an interesting perspective of knowledge graphs from the point of view of knowledge management. A key issue addressed in the knowledge management community is that of knowledge creation and the distinction between explicit knowledge and implicit or tacit knowledge. The widely accepted view is that the process of knowledge creation relies on the transformation of implicit knowledge into explicit knowledge. The goal of the paper is to study how knowledge graphs play a role in this conversion of implicit to explicit knowledge.

Section 2 presents a introduction to the knowledge management perspective and focus on the distinction between data, information and knowledge. This part of the paper is without clear focus as it is unclear what the main points are. It seems to me the main point is that knowledge increases the capacity to act, so knowledge is related to action. The data/informaiton/knowledge distinction is without focus as the relationship to Sematnic Web is unclear because the discussion states that information is about semantics, so it suggests that Semanatic Web is about representing informaiton rather than knowledge. This point is confusing.

The subsetion on explicit vs explicit knowledge is very interesting as the perspective is that explicit knowledge is written down, and this can be in any form. The paper misses an opportunity here to relate to Semantic Web. According to this definition, text documents, videos and images carry explict knowledge, and so do spreadsheets, RDF statements, etc. Implicit knowledge is in people's heads and organization procedures. This is again very interesting because the knowledge managment perspective makes no distinction between the format of explict knowledge. There is a strong sense that the Semantic Web community does not hold this point of view. This ought to be discussed.

Sections 2.2 and 2.3 on knowledge creation are also interesting, and again the relation to Semantic Web is lost on me.

At the end of section 2 I am left with an unanswered question. In the KM perspective explicit knowledge is knowledge that is written down, and it seems to me that knowledge can be written as text documents, multi-modal documents, some form of graph representation (RDF or property), or even as a collection of floating point numbers (vectors or parameters of deep neural nets). These are all written down and thus qualify as explicit knowledge. A discussion is needed to understand whether according to the KM perspective these are all valid representations of explicit knowledge, and whether the KM perspective considers any of them to be preferred. It seems to me that a text document is a very effective representation of knowledge because according to the capacity to act definition, a text document is perfectly adequate as the reader will know how to act after reading it. The other representations seem to require software for interpretation.

Section 3 is a summary of knowledge graphs. Introducing knowledge graphs before section 2 would allow section 2 to make references to knowledge graph and provide a more insightful description of the knowledge management perspective by contrasting it to the knowledge graph perspective.

I didn't find section 4.1 particulary insightful as it does not discuss any relationship to the KM perspective and the title of the paper "how much knowledge is in KGs". This section could be shortened with no loss of continuity for the paper. Alternatively, it could be expanded to relate it to the main objective of the paper.

The case studies section is the most interesting section. The authors study the knowledge generation contirbution of the three use cases. I am not convinced that the three use cases represent three different uses as all of them contribute to generation of knowledge in the same way. Tacit knowledge (latent relationships between the multiple KGs or data sources used) are made explict by representing them as links in KGs. One can view entity resolution and schema mapping (both making explicit tacit knowledge) as the creation of links between previously unconnected (tacit) entities/items.

This section should be revised to link back to the KM definition/framework of knowledge generation (tacit to explicit) making it clear how KG technology in each use case is performing the tacit to explicit transformation.

The argument for C1/C2/C3 in the conclusions is unconvincing, although the overall idea of the KM perspective is very interesting and very welcome. I recommend refocusing on the tacit to explict distinction as it is the fundamental aspect of knowledge creation.

In my opinoon the conclusions section misses the opprotunity to discuss recent developments in KGs from the KM perspective. Particularly, the idea whether millions of floating point numbers constitute an explict representation of knowledge. On the one hand, they are explict as they are written down; on the other hand one needs a special tool to read them (a deep neural net that makes predicions). The point is that these numbers can be used to convert tacit to explcit knowledge in the sense that the prediction of a neural net are explict (eg links with probabilities). I think the KM perspective would offer an interesting view on the ML methods and enhance the contribution of the paper.

In summary, I very much like the idea of the paper; I think the execution could be better, offering crisper insights into the KM perspective. Sectons 4 and 5 need significant revisions.

1) originality: does the paper convey new ideas or perspectives?
The knowledge management perspective is a new perspective for studying KGs. The perspective is new, but its execution in the paper could be better, as discussed in the main review.

2) quality of writing
The quality of the writing is good; the paper is easy to follow and overall, the writing is scholarly. The main review has a few suggestions to reorganize the sections to improve the flow.

3) relevance to the community: is the proposed topic relevant to the Semantic
Web community
The topic (knowledge creation) is relevant to the Semantic Web community, but there are no clear take-aways. The conclusions C1/C2/C3 and the new directions D1/D2/D3 are very high level. I wish there were some metrics or experiments where one could measure the extent to which the Semantic Web community is consistent/inconsistent with the KM perspective, which would help to produce actionable recommendations. Deeper analyses of use cases could provide similar insight. In its current form, relevance to the Semantic Web community is tangential.

4) potential impact to open up new research directions: in how far are open
research questions and directions mentioned and how relevant are these?
The impact is low as the new directions D1/D2/D3 are high level, and not entirely new. For example, commonsense knowledge graphs have a significant focus on actionable knowledge

5) interdisciplinarity: in how far does the paper enable/foster
interdisciplinary discussions connecting to other communities?
The paper offers a connection to the KM community, but it is unclear how the connection would evolve. A connection needs shared artifacts of some sort that the communities study or use in their work. These can be resources (KGs), tools, algorithms, data. This part is not defined.

This paper addresses the very relevant and interesting question of identifying the actual role of knowledge in current Knowledge Graphs. The paper provides a nice overview for the definition of what Knowledge is in the first place (which is very useful), and how this notion is related to Knowledge Graphs, as one of the most prominent current knowledge representation formalism.

The authors then proceed to study a number of papers from the industry track of ISWC (2017-19), and the role that KGs play in the creation of new knowledge.

I find the approach and contribution of the first half of the paper very timely and useful, but consider the actual contribution too restricted to warrant publication in a Journal in its current state. While the first 3 chapters are all very general, and (positively) ambitious, the second part of the submission is very restricted and narrow. Given the generic nature of the research question in a journal, the focus of three years of a single track of a single conference is in my view insufficient. But even more problematic is the choice to only go into detail of the analysis of 3 papers, in addition to a rather shallow analysis of the remaining 45 papers (or even the 16 papers about KGs for Knowledge creation.

As I, finally, also consider the structure of the paper rather confusing, I cannot recommend this paper for publication. Instead, I would recommend that the authors to resubmit an improved version of this paper to a conference, first, and then extend the analysis significantly across various dimensions (number of papers, conferences, years, aspects, etc).

In a bit more detail: I have not much to say about chapters 1-3, which I found rather informative, and well written. Given that this is a literature overview, it might make sense to change the citation style to one including author names (consistently). In its current form, there are too many references by number only, it might help to name the authors to make the paper more readable.

My real problems start in Chapter 4. While before the authors stressed the generality of the problem AND the notion of KGs, there is a sudden restriction to a single venue and few years. Not only are there other venues in the domain with industrial contributions (ESWC, Semantics, TheWebConf, to name just a few), but also have KGs become standard in other research communities. While a small scale qualitative analysis might be useful, it is too restricted for the claims of the first part of the paper, and the role of an archival Journal.

The order of the paper does not help, as the paper seems a bit upside-down. In Section 5, a number of interesting categories are identified, and 3 further dimensions which are labelled as further research. For me, those constitute interesting dimensions with which the papers could be analyzed, it is not clear to me why they are presented as summary.

The least that I would have expected would have been a detailed analysis of all the 16 papers that the authors considered relevant for knowledge creation. Then, the three types of use-cases could have been the result of the analysis, with commonalities and differences discussed in detail. In the paper itself, these three types are first taken as givens in order to choose the 3 chosen papers, and then as conclusions. I do not understand that presentation-logic.

I honestly do not understand whether the choice of these three classes is the result or the assumption of the analysis in the paper.

Some minor comments:
P3L20: have a long -> have lon
P4C2L11: in addition to the those.
P4C2L19 involves a different pair combination
P5C2L19: hav been constructed from human collaboration (from?)
P6C2L2: in our opinion: that should be argued in a Journal paper.
P6C2L4: and our the respective
P6C2:33: from the 48 papers you choose 20 on the basis of occurrence of the term KG. Would it have made sense to also look at the other 28 papers? The number is small enough that a more thorough analysis could have taken place.
There could also have been an analysis over time: did the number increase?
P7C1L9: you should link to something more archivable than a spreadsheet, I would think.
P7C1L18: you claim that in your opinion that the papers are representative for the three use-case types. I think that this is the wrong way round, why not first analyse the 20 papers and identify the types.
P7 use-cases: it would make sense to align the criteria on P6C2 with the ones in table 2 and the case-studies. Here, they have 3 different names, and differ in the table on P8, the use-case on P7 and the list on P6.
Finally, P9, C1 where do the research directions come from? They seem rather unrelated to the analysis of the research papers.

> 1) originality: does the paper convey new ideas or perspectives?

The paper is a review paper about trying to measure the amount of knowledge in practical applications, and as such relatively novel. The ambition clearly conveys a new angle to the usage of semantics, but unfortunately the execution in the paper falls short of the promise made.

> 2) quality of writing

The paper is pleasantly written with only minor editing problems. The second part of the paper, starting from section 4, though is not well structured in my opinion. The role of the 3 Case studies is not well motivated and only becomes more logical after one has read Section 5.

> 3) relevance to the community: is the proposed topic relevant to the Semantic Web community

In my view, the paper is relevant for the SW community, even though I have a small reservation: if the paper would provide a true and ambitious analysis of the amount of knowledge in practical applications of knowledge graphs, the focus could be even broader, as could the intended audience. Because of the focus in the analysis on proceedings of a SW conference, the paper is an immediate fit. But also a broader view would still fit the SWJ very well.

4) potential impact to open up new research directions: in how far are open research questions and directions mentioned and how relevant are these?

The paper brings up an interest research question, and as such identifies an interesting future research direction. A more quantitative approach to the analysis of the true amount of knowledge in applications of knowledge graphs could be very valuable, and sure relevant for the community. The concrete implementation of this research plan in the paper itself, though, has only limited impact in its current form.

5) interdisciplinary: in how far does the paper enable/foster interdisciplinary discussions connecting to other communities?

The paper itself does not provide any view on interdisciplinary, apart from the more introductory parts which suggest that the questions posed by the authors should ideally be addressed by interdisciplinary teams (economists, philosophers, organisation scientists). This is, however, not pursued in the remainder of the paper.

1) originality: does the paper convey new ideas or perspectives?

Yes - I think the paper adds to the field.

I should mention that the tacit knowledge vs. explicit knowledge split has moved on a little. Tacit versus explicit is not seen so binary any longer, they appear somewhat on a continuum. The purists argue that tacit knowledge is precisely that - tacit. However more practical KM researchers argue that tacit knowledge can be articulated to varying degrees and is useful obviously in the workplace and life in general. Furthermore it is not possible to make use of articulated knowledge without understanding the tacit background. The two work in unison - they are not separate bodies of knowledge. At the underlying level one needs tacit knowledge to appreciate the codifed set of knowledge. Ultimately knowledge becomes formalised in formula - highly descriptive. As you point out wisdom may be judicious use of knowledge, using human judgement and you keep this outside the scope of your paper.

"It also showed that technology proves as valuable means ..." can this be expanded upon please? It is not clear what is going here?

Table 1 - can this be extended further back as 3 years is minimal? Three years is ok, but I wonder why only such a short period of time?

Under Categorisation and metrics - 'collected a set of - in our opinion - relevant features and metrics' - can this point be clarified please - what made them so?

Under creation of new knowledge - where established links is mentioned the system allows advanced queries - some examples of these advanced queries could be providedd - at least one.

I like the examples such as football whispers.

2) quality of writing

The quality of writing is good, but one can tell it has not been written by native English speakers, however this issue is very minor. If the authors can get a native English speaker to correct the expression it will be all the stronger for it.

3) relevance to the community: is the proposed topic relevant to the Semantic Web community

I think so, but I do not belong to the Semantic Web community - I an in information systems and more specifically knowledge management.

4) potential impact to open up new research directions: in how far are open
research questions and directions mentioned and how relevant are these?

There are one or two research questions, but this is not a scientific research paper with hypotheses and postitivist testing regimes etc.

The paper fits in the computing space broadly and proposes a novel means by which graphs and more specifically directed graphs may be used to may flow flows for organisational benefit. The formal methodology (and the research epistemology underlying the study) is not so explicitly covered, but this is not unusual for a computing paper.

5) interdisciplinarity: in how far does the paper enable/foster
interdisciplinary discussions connecting to other communities?

Yes I think the paper is readable across disciplinary boundaries and provides a novel technique (graphs) which can help in knowledge management.

Perhaps a suggestion would be to discuss some of the organisations where the work presented here could be best made use of - are there particular industries where the knowledge graphs as a concept would be most useful and then give some concrete examples of how? This is a suggestion only as the authors have done a reasonable job adding the KM field. The latter discipline is growing and new approaches can only help. Time will tell if KGs help the KM domain.

1) originality: does the paper convey new ideas or perspectives?

The research question of this study is stated as "How can knowledge graphs foster the conversion of implicit into explicit knowledge and thus support the generation of new knowledge in organisations?"

-The paper attempts to present an interesting concept of knowledge graphs. That said, it is not quite clear what is a knowledge graph and knowledge graph technologies? What is the contextual focus here?. Can tacit knowledge or know-how be converted using a knowledge graph? If so, how this can be explained and justified in the study?

2) quality of writing

The authors suggest "We herein aim at examining how KGs could become the key to make an organisation’s knowledge explicit and how they can provide us with possibilities to identify, combine, and make sense of existing (internal) organisational knowledge and given resources in order to create new knowledge."

Later in Section 5 on Summary and Conclusions, the authors claim that “We have herein investigated how KG technologies indeed support organisations and companies in the process of knowledge creation. How knowledge is created has been researched extensively in the KM community.”

-There are parts of the text where the flow of argumentation could be refined with a better focus and consistency.

-For example, in the above the focus of the study aim is somewhat different from the study’s RQ.

-In other words, RQ emphasizes on how KG foster conversion of implicit to explicit and support knowledge generation. Whereas the study aim focus is to identify, combine and sensing of internal organisational knowledge.

-So, it is not adequately clear to the reader what is the precise focus of the study and in which specific context/s?

-Overall, the paper needs more clarity and consistency of focus between the research question, study aim and summary/conclusions.

3) relevance to the community: is the proposed topic relevant to the Semantic Web community

-Not sure who is the audience of the Semantic Web Community? Considering my research expertise is not in this domain of knowledge, I am unable to comment on this point.

4) potential impact to open up new research directions: in how far are open research questions and directions mentioned and how relevant are these?

-In the KM research literature, there is potential for a contribution.

5) interdisciplinarity: in how far does the paper enable/foster interdisciplinary discussions connecting to other communities?

-Evidence of interdisciplinary discussions need articulation and development