https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/Head https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://www.nanopub.org/nschema#hasAssertion https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/assertion https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://www.nanopub.org/nschema#hasProvenance https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/provenance https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://www.nanopub.org/nschema#hasPublicationInfo https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/pubinfo https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.nanopub.org/nschema#Nanopublication https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/assertion http://id.crossref.org/issn/2451-8492 http://purl.org/dc/terms/title Data Science https://doi.org/10.3233/DS-190019 http://purl.org/dc/terms/abstract Context: Systematic Reviews (SRs) are means for collecting and synthesizing evidence from the identification and analysis of relevant studies from multiple sources. To this aim, they use a well-defined methodology meant to mitigate the risks of biases and ensure repeatability for later updates. SRs, however, involve significant effort. Goal: The goal of this paper is to introduce a novel methodology that reduces the amount of manual tedious tasks involved in SRs while taking advantage of the value provided by human expertise. Method: Starting from current methodologies for SRs, we replaced the steps of keywording and data extraction with an automatic methodology for generating a domain ontology and classifying the primary studies. This methodology has been applied in the Software Engineering sub-area of Software Architecture and evaluated by human annotators. Results: The result is a novel Expert-Driven Automatic Methodology, EDAM, for assisting researchers in performing SRs. EDAM combines ontology-learning techniques and semantic technologies with the human-in-the-loop. The first (thanks to automation) fosters scalability, objectivity, reproducibility and granularity of the studies; the second allows tailoring to the specific focus of the study at hand and knowledge reuse from domain experts. We evaluated EDAM on the field of Software Architecture against six senior researchers. As a result, we found that the performance of the senior researchers in classifying papers was not statistically significantly different from EDAM. Conclusions: Thanks to automation of the less-creative steps in SRs, our methodology allows researchers to skip the tedious tasks of keywording and manually classifying primary studies, thus freeing effort for the analysis and the discussion. https://doi.org/10.3233/DS-190019 http://purl.org/dc/terms/date 2019-08-19 https://doi.org/10.3233/DS-190019 http://purl.org/dc/terms/isPartOf http://id.crossref.org/issn/2451-8492 https://doi.org/10.3233/DS-190019 http://purl.org/dc/terms/title Reducing the effort for systematic reviews in software engineering https://doi.org/10.3233/DS-190019 http://purl.org/ontology/bibo/issue 1-2 https://doi.org/10.3233/DS-190019 http://purl.org/ontology/bibo/volume 2 https://doi.org/10.3233/DS-190019 http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://purl.org/spar/fabio/ResearchPaper https://orcid.org/0000-0001-6365-6515 http://schema.org/affiliation https://ror.org/01j9p1r26 https://orcid.org/0000-0001-6365-6515 http://schema.org/email henry.muccini@univaq.it https://orcid.org/0000-0001-6365-6515 http://xmlns.com/foaf/0.1/name Henry Muccini https://orcid.org/0000-0001-6557-3131 http://schema.org/affiliation https://ror.org/05mzfcs16 https://orcid.org/0000-0001-6557-3131 http://schema.org/email francesco.osborne@open.ac.uk https://orcid.org/0000-0001-6557-3131 http://xmlns.com/foaf/0.1/name Francesco Osborne https://orcid.org/0000-0002-2234-0845 http://schema.org/affiliation https://ror.org/008xxew50 https://orcid.org/0000-0002-2234-0845 http://schema.org/email p.lago@vu.nl https://orcid.org/0000-0002-2234-0845 http://xmlns.com/foaf/0.1/name Patricia Lago https://orcid.org/0000-0003-0015-1952 http://schema.org/affiliation https://ror.org/05mzfcs16 https://orcid.org/0000-0003-0015-1952 http://schema.org/email enrico.motta@open.ac.uk https://orcid.org/0000-0003-0015-1952 http://xmlns.com/foaf/0.1/name Enrico Motta https://ror.org/008xxew50 http://xmlns.com/foaf/0.1/name Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands https://ror.org/01j9p1r26 http://xmlns.com/foaf/0.1/name DISIM Department, University of L’Aquila, Italy https://ror.org/05mzfcs16 http://xmlns.com/foaf/0.1/name Knowledge Media Institute, The Open University, UK https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list http://www.w3.org/1999/02/22-rdf-syntax-ns#_1 https://orcid.org/0000-0001-6557-3131 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list__1 http://www.w3.org/1999/02/22-rdf-syntax-ns#_2 https://orcid.org/0000-0001-6365-6515 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list__2 http://www.w3.org/1999/02/22-rdf-syntax-ns#_3 https://orcid.org/0000-0002-2234-0845 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list__3 http://www.w3.org/1999/02/22-rdf-syntax-ns#_4 https://orcid.org/0000-0003-0015-1952 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/provenance https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/assertion http://www.w3.org/ns/prov#wasAttributedTo https://orcid.org/0000-0001-6365-6515 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/assertion http://www.w3.org/ns/prov#wasAttributedTo https://orcid.org/0000-0001-6557-3131 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/assertion http://www.w3.org/ns/prov#wasAttributedTo https://orcid.org/0000-0002-2234-0845 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/assertion http://www.w3.org/ns/prov#wasAttributedTo https://orcid.org/0000-0003-0015-1952 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/pubinfo https://orcid.org/0000-0001-6365-6515 http://xmlns.com/foaf/0.1/name Henry Muccini https://orcid.org/0000-0001-6557-3131 http://xmlns.com/foaf/0.1/name Francesco Osborne https://orcid.org/0000-0002-1267-0234 http://xmlns.com/foaf/0.1/name Tobias Kuhn https://orcid.org/0000-0002-2234-0845 http://xmlns.com/foaf/0.1/name Patricia Lago https://orcid.org/0000-0003-0015-1952 http://xmlns.com/foaf/0.1/name Enrico Motta https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/dc/terms/created 2025-05-26T09:28:03.727Z https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/dc/terms/creator https://orcid.org/0000-0002-1267-0234 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/dc/terms/license https://creativecommons.org/licenses/by/4.0/ https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/nanopub/x/hasNanopubType http://purl.org/spar/fabio/ScholarlyWork https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/nanopub/x/hasNanopubType https://w3id.org/kpxl/ios/ds/terms/DataScienceNanopub https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/nanopub/x/introduces https://doi.org/10.3233/DS-190019 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/nanopub/x/wasCreatedAt https://nanodash.knowledgepixels.com/ https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://purl.org/ontology/bibo/authorList https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc http://www.w3.org/2000/01/rdf-schema#label Article: Reducing the effort for systematic reviews in software engineering https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromProvenanceTemplate https://w3id.org/np/RAekcN47h13fk6ZK4XiObgGgk-qB01sLOjyGyhMCq_jT4 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromPubinfoTemplate https://w3id.org/np/RA0J4vUn_dekg-U1kK3AOEt02p9mT2WO03uGxLDec1jLw https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromPubinfoTemplate https://w3id.org/np/RA16U9Wo30ObhrK1NzH7EsmVRiRtvEuEA_Dfc-u8WkUCA https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromPubinfoTemplate https://w3id.org/np/RA5R_qv3VsZIrDKd8Mr37x3HoKCsKkwN5tJVqgQsKhjTE https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromPubinfoTemplate https://w3id.org/np/RAYrlN4s93vVe9LGI-gmPLTb-QZGHKd0mxx8VxJ3XVhuw https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromPubinfoTemplate https://w3id.org/np/RAukAcWHRDlkqxk7H2XNSegc1WnHI569INvNr-xdptDGI https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/np/o/ntemplate/wasCreatedFromTemplate https://w3id.org/np/RAdU0hUGGpPxWxtO3n_c4loIpyd67K5_Tr7w5ul8sA-iY https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list http://www.w3.org/1999/02/22-rdf-syntax-ns#_1 https://orcid.org/0000-0001-6557-3131 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list http://www.w3.org/1999/02/22-rdf-syntax-ns#_2 https://orcid.org/0000-0001-6365-6515 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list http://www.w3.org/1999/02/22-rdf-syntax-ns#_3 https://orcid.org/0000-0002-2234-0845 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/author-list http://www.w3.org/1999/02/22-rdf-syntax-ns#_4 https://orcid.org/0000-0003-0015-1952 https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/sig http://purl.org/nanopub/x/hasAlgorithm RSA https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/sig http://purl.org/nanopub/x/hasPublicKey MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQD4Wj537OijfOWVtsHMznuXKISqBhtGDQZfdO6pbb4hg9EHMcUFGTLbWaPrP783PHv8HMAAPjvEkHLaOHMIknqhaIa5236lfBO3r+ljVdYBElBcLvROmwG+ZGtmPNZf7lMhI15xf5TfoaSa84AFRd5J2EXekK6PhaFQhRm1IpSYtwIDAQAB https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/sig http://purl.org/nanopub/x/hasSignature 7Y0vQHWAWlMu7SVqX0D+41hD+j4tdCUZFFTflIXa4LKn7MJvMRdImdY7gUP2W3G9vAr4Df+KE8Vkf44r5Gzs7+efcQbnnIh5uZMC6mQ/DDJ3AcbSur6J2BIDPKNmJn+NtcocUedfLkJPylgUBCk5TnUKyl0+paiQzL/STVP4MTI= https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/sig http://purl.org/nanopub/x/hasSignatureTarget https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc https://w3id.org/kpxl/ios/ds/np/RAWu9Hx_V1AD8Yyl1iad-2jIXWX8GkXhWG-4wEV4YaUQc/sig http://purl.org/nanopub/x/signedBy https://orcid.org/0000-0002-1267-0234