New SWJ Policy about Addressing Review Criteria using Supplemental Files

Since introducing paper types for ontologies, datasets, tools, and applications, we have tried to strike the right balance between asking authors to provide evidence for importance, usefulness, relevance, stability, and impact (as these are some of the criteria by which we ask reviewers to rate submissions) on the one hand and readability for a broad target audience on the other hand. Based on our observations with the current setup, we decided to adjust the balance by giving more weight to meeting the interests and needs of our readers.

For instance, while an in-depth explanation of the long-term maintenance strategy for a particular dataset is highly relevant during the review process, it becomes less relevant for future readers. As reviewers (rightfully) request details about stability, maintenance, evidence of importance, design decisions, and so forth, authors have increasingly started to focus more on these aspects compared to proving an interesting, informative, and well-illustrated paper that introduces an ontology, dataset, or tool to a broader audience.

From now on, it will be possible to address review criteria such as importance, usefulness, relevance, stability, and impact in two ways, namely within the submission or as publicly visible supplemental files published on the journal's webpage. Authors that decide to submit supplemental files are nonetheless asked to briefly comment on all review criteria within their submissions. By doing so, we ensure that there will be a visible provenance record documenting why a paper met the review criteria and at the same time hope to reduce the burden this has put on the overall readability of accepted papers.