Taxonomy Enrichment with Text and Graph Vector Representations

The authors address the problem of taxonomy enrichment which aims at adding new words to an existing taxonomy. Deep representations of graph structures like GCN autoencoder, Poincaré embeddings, node2vec have recently demonstrated very promising results on various NLP tasks. The authors' approach is a comprehensive study of the existing approaches to taxonomy enrichment based on word and graph vector representations. They also explore the ways of using deep learning architectures to extend taxonomic backbones of knowledge graphs. They achieve state-of-the-art results across different datasets.

(1) Originality

The work is fairly original. The authors present a computational study of various approaches
to taxonomy enrichment including recent
state-of-the-art results, datasets for studying diachronic evolution of wordnets for English and Russian, and efficient methods for taxonomy enrichment.

(2) Significance of the results

I am divided on the significance of the results/potential impact. While their study shows a few percentage point improvements, the combinations used are fairly simple and expected. That being said, I still think this is a result that deserves to get published.

(3) Quality of writing

The paper was fairly well-written with some minor typos toward the end i.e. the caption of Figure 4 ('streamlinign') and also on page 23 ('compelmentary'). If the authors can make the paper a bit shorter, that would also be better.

This paper is about "taxonomy enrichment" based on text and graph vector representations. This paper is quite long, really badly organized, with a lot of experimental results but not any clear line.
The objectives and the subsequent organization of this paper are among the main problems. First of all, what is exactly the objective of this paper? Is this paper a survey or a paper about a research topic, maybe to build a "taxonomy" for understanding and managing Russian texts? This is absolutely not clear.
This is really a problem when reading this paper as when arrived at the end, the reader does not exactly know what should be understood and what is the lesson learned.
In addition, as a submission to the Semantic Web journal, the reader wonders again what is the relation with Semantic Web, again this is absolutely not clear.

In the paper, many things until section 6 are introduced and the organization of the texts is always between specific topics and survey, the separation line is never clear and this makes the reading and the understanding very hard.
Many elements are introduced making a parallel between English and Russian languages, but there are no concrete and substantial examples for illustrating the purposes. All examples which are proposed are very specific and do not help to have a global view of the content of this research.
Moreover, the paper is about "taxonomy enrichment". First what is exactly a "taxonomy" here and what "taxonomy enrichment" is meaning? The term "taxonomy" is never defined and instead is taken as understood, but it would be better to know what is exactly the underlying mathematical structure.
Then what do you mean by "enrichment" and what is the purpose of this "taxonomy enrichment"? This is very quickly mentioned in the introduction but no more made precise and discussed in the rest of the paper.
In addition, as we are reading a submission for the semantic Web journal, we could expect that this "enrichment operation" is performed w.r.t. a given semantics and that the principles of knowledge representation may have a certain importance here. Nothing at all is mentioned about that and any preoccupation about semantics is totally absent. We could expect a discussion on this subject as such an automatic enrichment of a hierarchy of terms recalling a TBox in an ontology could and maybe should conform in certain sense to a given semantics, but nothing is said about that.

The authors should carefully review their article, maybe shorten it, make a clear separation between what is related to state of the art and what is related to their research problem. They should pose in a clear manner their own problem, from the theoretical and practical point of view, give justifications and illustrations of what they are intending to do.
They should explain how experiments are showing and justifying their objectives, and keep in the results only what is important, separating important tables from the other ones that could be placed in appendices or even removed.
Moreover, and very importantly, they should also discuss the applicability and generality of their research work, its reusability and its generality, for example to which extent we can apply this work to another language?
Finally, a justification of why this paper is submitted to the Semantic Web journal is also expected.

The paper presents a comprehensive study of several approaches for taxonomy enrichment and an error analysis describing their typical mistakes. To this purpose, the authors introduce two new datasets for this task extracted by WordNet (in English and Russians) and evaluate alternative solutions based either on word representations or graph representations

The paper is well written and clear. The topic is very relevant to the special issues.

This is a very interesting paper that can be a good starting point for future research in the area. In particular, I appreciate Section 7 which does an in-depth error analysis on the different types of mistakes produced by state-of-the-art approaches and offers some interesting insights of their limitations. The most interesting finding is that the methods based on word vector representation appear to outperform graph-based approaches, which are usually performing well on link prediction tasks.

The only major issues of the paper regard the sharing of data and the contributions with respect to previous articles, which I would like to clarify before acceptance.

A good portion of the paper appears to derive from the article:
Irina Nikishina, Varvara Logacheva, Alexander Panchenko, Natalia V. Loukachevitch: Studying Taxonomy Enrichment on Diachronic WordNet Versions. COLING 2020: 3095-3106

While it if fine for the paper to be an extension of COLING 2020, the relationship between the two paper should be stated explicitly. In particular the authors should clarify in the introduction 1) if the paper is an extension of COLING 2020, and 2) what contributions are new.

Regarding the sharing of the data, I commend the authors for sharing code and data regarding the two novel datasets. However, it is not clear to me if the authors are sharing all code and data regarding the study presented in the paper. The linked Github repo (https://github.com/dialogue- evaluation/taxonomy- enrichment) seems to contain only part of the data, but I may be wrong. The authors should make all the data available and clarify better where they can be found.

In the following, please find additional comments/questions on specific sections.

Section 2
The state of the art is quite comprehensive. I would suggest to briefly refer to methods for extracting taxonomies/ontologies/knowledge graphs from the text, such as “Cimiano, P. and Völker, J., 2005. Text2Onto. In International conference on application of natural language to information systems. Springer, Berlin, Heidelberg.”, “Luan, Y., He, L., Ostendorf, M. and Hajishirzi, H., 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. arXiv preprint arXiv:1808.09602.”,“Dessì, D. et al. 2021. Generating knowledge graphs by employing Natural Language Processing and Machine Learning techniques within the scholarly domain. Future Generation Computer Systems, 116”. It may be useful to mention also some of unsupervised approaches developed by the semantic web community such as “Osborne, F. and Motta, E., 2015. Klink-2: integrating multiple web sources to generate semantic topic networks. In International Semantic Web Conference 2015. Springer, Cham” and “Han, K. et al. 2020. WikiCSSH: extracting computer science subject headings from Wikipedia. In ADBIS, TPDL and EDA 2020 Common Workshops and Doctoral Consortium. Springer, Cham.”

Section 4
“This baseline has a shortcoming […]” This reads confusing at the beginning of a new section. I would rephrase with something such as “The solution presented in the previous section has…”

“This approach was presented at the RUSSE’2020 33 Taxonomy Enrichment task” Why is this approach not cited?

Section 6
The evaluation would be stronger if a statistical significance test was presented, in particular for validating some of the major findings of the study, such as the difference between word- based and graph-based methods.

Typos
“non-linear projection transofrmations” > transformations