A comparison of object-triple mapping frameworks

Tracking #: 1910-3123

This paper is currently under review
Authors: 
Martin Ledvinka
Petr Křemen

Responsible editor: 
Philippe Cudre-Mauroux

Submission type: 
Survey Article
Abstract: 
Domain-independent information systems like ontology editors provide only limited usability for non-experts when domain-specific linked data need to be created. On the contrary, domain-specific applications require adequate architecture for data authoring and validation, typically using the object-oriented paradigm. So far, several frameworks mapping the RDF model (representing linked data) to the object model have been introduced in the literature. In this paper, we develop a novel framework for comparison of object-triple mapping solutions in terms of features and performance. For feature comparison, we designed a set of qualitative criteria reflecting object-oriented application developer's needs. For the performance comparison, we introduce a benchmark based on a real-world information system that we implemented using one of the compared OTM solutions -- JOPA. We present a detailed evaluation of a selected set of object-triple mapping libraries and show that they significantly differ both in terms of features and time and memory performance.
Full PDF Version: 
Tags: 
Under Review

Comments

We are the authors of the KOMMA framework.

Thank you for this interesting work which we are sure will help to further improve the performance of OTM solutions.

Regarding KOMMA we have the following remarks:

Features:

Performance:

KOMMA's data change tracker (required for correct undo/redo behaviour) needs to be disabled to achieve good performance in batch operations. Please use this code to disable it. (We can provide a better API to disable it in the future.) Otherwise any insertion or removal of a statement is validated with a query against the database.

  • OP1 − Create and OP2 − Create-Batch: Performance is largely improved by disabling the data change tracker.
  • OP5 − Update: The benchmark code needs to be modified according to this commit to correctly run the update within a transaction. In its current version the updates are run outside of the transaction. The use of "merge" is only required to move data between different entity managers.
  • OP3 − Retrieve and OP4 − Retrieve all: We are in the process of investigating the benchmark code. An issue is that a bean is invalidated if its properties are changed. Hence the benchmark loads the beans from the database AND reloads the created beans for comparison. Furthermore, KOMMA is able to load a whole object graph (in this case of a single OccurrenceReport) with:
    em.createQuery("construct { ?r a . ?s ?p ?o } where { ?r (!<:>|<:>)* ?s . ?s ?p ?o }")
    .setParameter("r", uri).getSingleResult(OccurrenceReport.class)

Additionally, we are not able to reproduce the performance figures of Alibaba. Maybe this is due to a difference in the repository configuration for GraphDB (We created the benchmark repository with the default settings from the workbench).

Dear reader of this comment thread,

We have been in contact with KOMMA developers so we post a reply here mainly for information of anyone reading Ken's original comment.

First of all, we thank the KOMMA developers very much for their insight into KOMMA. In order to make our benchmark repeatable for the paper readers, we based the benchmark setup of each tool on information available in its documentation and a limited investigation of its source code (about 5h) only. Unfortunately, none of the remarks posted by Ken were part of KOMMA documentation.

We kindly ask KOMMA developers to include the remarks into the documentation. Then - since we have already implemented the changes based on our communication with them - we are able to rerun the updated benchmark, include the results on the benchmark web and update the paper in its next revision.