Back
Texte intégral
Journal article

Testing machine learning based systems : a systematic mapping

  • Riccio, Vincenzo Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
  • Jahangirova, Gunel Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
  • Stocco, Andrea Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
  • Humbatova, Nargiz Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera ·
  • Weiss, Michael Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
  • Tonella, Paolo Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
Show more…
    2020
Published in:
  • Empirical Software Engineering. - Springer. - 2020, vol. 25, no. 6, p. 5193–5254
Systematic mapping
Systematic review
Software testing
Machine learning
English Context: A Machine Learning based System (MLS) is a software system including one or more components that learn how to perform a task from a given data set. The increasing adoption of MLSs in safety critical domains such as autonomous driving, healthcare, and finance has fostered much attention towards the quality assurance of such systems. Despite the advances in software testing, MLSs bring novel and unprecedented challenges, since their behaviour is defined jointly by the code that implements them and the data used for training them. Objective: To identify the existing solutions for functional testing of MLSs, and classify them from three different perspectives: (1) the context of the problem they address, (2) their features, and (3) their empirical evaluation. To report demographic information about the ongoing research. To identify open challenges for future research. Method: We conducted a systematic mapping study about testing techniques for MLSs driven by 33 research questions. We followed existing guidelines when defining our research protocol so as to increase the repeatability and reliability of our results. Results: We identified 70 relevant primary studies, mostly published in the last years. We identified 11 problems addressed in the literature. We investigated multiple aspects of the testing approaches, such as the used/proposed adequacy criteria, the algorithms for test input generation, and the test oracles. Conclusions: The most active research areas in MLS testing address automated scenario/ input generation and test oracle creation. MLS testing is a rapidly growing and developing research area, with many open challenges, such as the generation of realistic inputs and the definition of reliable evaluation metrics and benchmarks.
Language
  • English
Classification
Computer science and technology
License
CC BY
Open access status
hybrid
Identifiers
Persistent URL
https://n2t.net/ark:/12658/srd1319375
Statistics
Document views: 122 File downloads:
  • Riccio_ese_2020.pdf: 168