|Review Comment: |
The paper at hand is a survey article about Machine Translation (MT) approaches that rely on Semantic Web Technologies (SWT).
The topic is very pertinent to both the Semantic Web community and the MT community, and is a hot research topic.
The main claim of the authors is that SWT can be used to solve the problem of lexical and semantic ambiguity, which is one of the main problems in MT, and that the potential of these technologies is “still in its infancy”.
The paper is organized in the following way. In section 2 the authors explain the research methodology followed to identify papers in which approaches that use SWT in MT are described. The result is a list of 17 papers whose authors and titles are included in Table 1. In order to classify each study according to the translation approach they follow, they provide a classification of MT approaches in section 3, and describe the main types of MT approaches in the subsequent subsections. Then they identify the challenges of such approaches and compare them. Section 4 consists of several sections. They start by providing a brief overview of the possible benefits of applying SWT to MT systems. Then, they describe each of the studies surveyed according to the MT approaches identified in section 3. Finally, they discuss and suggest future solutions to 3 MT challenges: Disambiguation, Non-standard speech, and Named Entity Recognition. In section 5 they conclude the paper.
The paper needs a thorough review of the English (specially, section 4).
One of my concerns with the paper, however, is the lack of systematization and consistency in some sections that need to be systematic and consistent (specially, again, section 4). Also, the accuracy and precision in the use of some concepts, dimensions, classifications schemas, etc., used along the paper. I have included a detailed justification of these concerns in the rest of the review. Specifically, in the sections that describe the identified studies for this survey, I miss a clear and concise description of each of them. Sometimes the information is not clearly presented or structured, some items of information are not clearly related, and the main aspects of the studies not appropriately highlighted (more as a copy-paste of unrelated sentences). I miss a lack of systematization in the description of the systems (maybe they could use the four dimensions introduced in section 3.1?). This should be solved if the paper is to be accepted for publication. Finally, the relevance of some of the approaches is also doubtful, since some have not been appropriately evaluated.
As for the introduction, could they provide more specific data of the types of errors? They claim that WSD is the “most common source of error”. What about other errors such as inflectional errors, reordering errors, missing words, etc.? Popovic and Ney (2011) in their article in Computational Linguistics provide a classification of typical errors in MT (also Vilar et al. 2006). It would be very interesting to have some numbers or percentages of the incidence of the different types of errors. Or are they just focusing on how SWT can help in solving the WSD problem in MT?? (then maybe the scope of the paper should be re-thought).
As for Corpus-Based Machine Translation (CBMT) approaches, they say that the problems of these approaches are “connected to the problem of ambiguity, including syntactic variations, expressions, irregular verbs, slang, and others”. Do they mean that “syntactic variations, expressions, irregular verbs, slang, and others” are subtypes of ambiguity problems? I don’t think this is accurate.
In section 2 they start by presenting the research question they aim at answering with this survey, which is: How can SWT enhance MT quality? Then, they formulate 4 more questions that are subsumed by this one. Two of them mention explicitly Linked Data (RQ1. What are state-of-the-art approaches in MT which use “Linked Data”? And RQ4. What kinds of Linked-Data-driven tools are available for MT?) In the other two, they refer to SWT and ontological knowledge. I think that they should justify the interest of these questions. Also, Why do they use Linked Data in some of them, and why ontological knowledge and SWT in others? Do they mean something different by each of these terms? Isn’t the use of Linked Data too restrictive ein RQ1 and RQ4?
As for the criteria for identifying a study and including it in their survey, they claim that studies should “focus on the evaluation of multilingual approaches using Linked Data”. What is the interest of this criterion? What is meant by “multilingual approaches”? MT systems, or also other systems? I could think of several studies that would meet this criterion and that have not been included in the survey.
The forth criterion they include is not clear either: “Studies that evaluated MT based on SW principles”. What are these principles and how do they influence evaluation? An explanation of this criterion is as well required.
In the Exclusion criteria, again I think that the use of Linked Data imposes an unnecessary restriction on MT approaches that may use some other type of SW technology.
Is the exclusion criterion “Studies that did not focus on Linked Data, MT or SW” correctly formulated? Would a study focusing on Linked Data be accepted?
As for the search queries they introduce in section 2.2.3, the authors should justify the interest of the keywords selected, as well as the need for two search queries (was the first query now enough? Did it not return relevant results?).
They list a set of conferences in which matched papers were presented. I was wondering if they include in their searches some dedicated workshops associated to those conferences, such as the “Multilingual Semantic Web” workshop or the “Semantic Web Technologies for Machine Translation” workshop, or if they just searched the papers in the proceedings of the corresponding main conference.
In the search steps, they say that “we excluded publications that are not in English or did not contain any reference to SW”. Is this correct? Shouldn’t it be SW and MT, to be exact?
In section 3 they provide a classification of MT approaches according to 4 dimensions. Descriptions of the dimensions are maybe too brief and many questions remain open. For instance, in the case of the dimension called “problem space addressed”, they try to illustrate this dimensions with some examples that are not clear. Why do we need deep linguistic rules for translating old Egyptian texts? Probably, because SMT is not possible (no corpus available). Why “translating large volume of text is best carried out using statistical method”? It will in its turn depend on the languages involved, the existence of parallel corpora to previously train the system, etc., right?
Then, to be systematic, one would expect to identify the 4 dimensions in the subsequent descriptions of MT approaches, but it is not the case. Specifically, “problem space addressed” and “performance” are not addressed in most of them.
As for CBMT, they mention that a bilingual corpus is needed… comparable corpora or a parallel one? Both? Details are needed.
Once the different approaches have been described, they include a section (section 3.3) in which some challenges are listed. Are these challenges general enough so that they are faced by all the systems reviewed? Or only faced by some of them? I miss many challenges there related to the different systems (corpus creation, superficial fluency, difficulties in extracting meaning of text and creating an interlingua…) A more systematic analysis is missed at this stage.
As for the final comparison, they only compare quality and run time behavior of RBMT and CBMT in the text, and do not take into account any other dimensions. On the contrary, they include a quite complete table of pros and cons of each approach. I would suggest they relate the text to the table, and explain with more details the information contained in the table.
Section 4.1 provides an overview of the possible benefits of applying SWT to MT. I do not understand the purpose of this section, nor its title. Moreover, some paragraphs need revision from both a content and a grammar perspective. For example in paragraph 2 they say, and I quote “Although the graph structure behind SW can act as a disambiguation method with high decision power, some SW concepts still need to be addressed before they can be applied successfully in MT systems”. What is meant by “some SW concepts”? Then they continue with the following sentence “For instance, the coverage of multilingual content in semantic structures and how to link this content with multilingual ontologies has to be improved”. Not clear what they mean. The same happens with the last paragraph in this section.
As for the first studies described in sections 4.2.1. and 4.2.2., I miss an organized description of the different steps in the translation process that involve the use of SWT. They are not systematic in the descriptions and sometimes take for granted that readers are experts in the field. For example, when referring to the different metrics (“so it may not be suitable as introductory text to get started on the covered topic”.)
Not clear how the approach in 4.2.3. works.
In Arcan et al. work, (section 18.104.22.168), what do you mean when saying that for translations the authors use the SKOS vocabulary? Skos is a model. What do you mean by “using SWT to retrieve ontology content resulted in clear improvements to the translation model”?
In McCrae and Cimiano’s work, what were the results of the evaluation? Not clear.
In Moussalem and Choren, what are the ontologies they use? Are all of them implemented in SKOS? Available implementations?
As for section 22.214.171.124, which MT approaches are described in there? Is BabelNet used in the MT system. The purpose of the section is not clear.
It the work by Vertan  an architecture or only a methodology? I think that the space devoted to this and related studies is unjustified.
In Almasound et al., what is “the ontology” used? Is it a domain ontology or an upper ontology (of general knowledge)?
In Knoth et al., what do they mean by this sentence “present a new approach combining CLIR and MT by multilingual domain ontologies”? How is the lightweight ontology generated?
The last paragraph describing the work by Simov et al. is not clear at all? Did they use DBpedia or LT4eL? WordNet or OntoWordNet? Also, MT metrics are included there, but have not been mentioned or briefly defined in previous sections.
What do they mean by “the algorithm performs WSD by analyzing the ontological relationships between parts of speech for potential translations”?
In section 4.3, could they provide more details on how SWT are applied to the output translation? Another question is how do they suggest that verbal tenses can be recognized by using relationships among properties? Shouldn’t they include a section about the use of reasoners in MT? How does this relate to the disambiguation challenge?
As for the NER challenge, they suggest the use of methods and tools to improve the results of NER, but do not relate it to the translation problem. How would this be integrated in the MT system? At which stage of the process?