On the use of semantic technologies for video analysis and retrieval

Tracking #: 1974-3187

Pierluigi Ritrovato
Mario Vento
Luca Greco

Responsible editor: 
Guilin Qi

Submission type: 
Survey Article
The rapid proliferation of video recording devices has led to a huge explosion of contents, determining an ever increasing interest towards the development of methods and tools for automatic video analysis and interpretation. Through the years, the availability of contextual knowledge has proven to improve video analysis algorithms' performances in several ways, although the formal representation of semantic content in a shareable and fusion oriented manner is still an open problem. In this context, an interesting answer has come from Semantic technologies, that opened a new interesting perspective for the so-called Knowledge Based Computer Vision (KBCV), adding new functionality, improving accuracy, and facilitating data exchange between video analysis systems in an open extensible manner. In this work, we propose a survey of the papers from the last seventeen years, back when first applications of semantic technologies to video analysis and retrieval have appeared. The papers have been analyzed under different perspectives to give an overview of the pursued approaches and semantic web technologies involved. As a result, some insights about current trends and future challenges are provided too.
Full PDF Version: 

Reject (Two Strikes)

Solicited Reviews:
Click to Expand/Collapse
Review #1
Anonymous submitted on 11/Oct/2018
Major Revision
Review Comment:

This manuscript was submitted as 'Survey Article' and should be reviewed along the following dimensions: (1) Suitability as introductory text, targeted at researchers, PhD students, or practitioners, to get started on the covered topic. (2) How comprehensive and how balanced is the presentation and coverage. (3) Readability and clarity of the presentation. (4) Importance of the covered material to the broader Semantic Web community.

Different from the first edition of this paper, I appreciate that most of my comments have been addressed. Similar with the first edition, this survey paper aims to show the semantic knowledge or contextual knowledge can be use both in video analysis and retrieval, and the semantic web technologies could improve the performance of existing solutions and enabling advanced video analytic functionalities in some ways. The contribution of this paper is give a comprehensive analysis of published works in the field of video analysis and retrieval. I appreciate that most comments of my original review have been addressed, not always to 100% satisfaction on my side. However, there still some issues should be considered in this paper:

1. The analysis of the relevant works is lack of comparison, and should give some experiment results.

2. This paper showed that the manual analysis in video analysis is time consuming and often misses some important information due to too many frames in video, then shows that vision-based methods is robust and reliable. Lastly, the paper introduces the semantic-based approaches in video analysis and retrieval field. However, the paper didn't give a comprehensive comparison between the traditional vision-based and the semantic-based approaches, which is inexplicable to the reader why this paper focus on semantic-based approaches rather than classical methods. So, it is very significant to show the importance and necessity of taking advantage of the semantic knowledge. Actually, a graphic example is good to point out the necessity of semantic knowledge in video analysis.

3. Different from the first version of this paper, the author added another taxonomy, i.e. video analysis. The essential differences between the three taxonomies of video analysis and video retrieval seem unclear. As I seen from the paper, the relevancy between video analysis and video retrieval are progressive relationship and there is intersection between video analysis and video retrieval. The architecture and the essential difference of the taxonomies in this paper should be future analysed.

Review #2
By Lingling Zhang submitted on 02/Nov/2018
Review Comment:

The authors have addressed all the issues in the revised manuscript. I have no comments and the paper is ready to be accepted.

Review #3
By Wankou Yang submitted on 10/Nov/2018
Major Revision
Review Comment:

This paper proposes a survey of Semantic Web technologies to video analysis and retrieval. This paper have the following problem:
Some comparable methods are old.There are many papers related to video analysis and retrieval. The authors should review and compare the in depth.