Testing machine learning based systems : a systematic mapping
-
Riccio, Vincenzo
Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
-
Jahangirova, Gunel
Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
-
Stocco, Andrea
Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
-
Humbatova, Nargiz
Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera ·
-
Weiss, Michael
Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
-
Tonella, Paolo
Facoltà di scienze informatiche, Università della Svizzera italiana, Svizzera
Show more…
Published in:
- Empirical Software Engineering. - Springer. - 2020, vol. 25, no. 6, p. 5193–5254
English
Context: A Machine Learning based System (MLS) is a software system including one or more components that learn how to perform a task from a given data set. The increasing adoption of MLSs in safety critical domains such as autonomous driving, healthcare, and finance has fostered much attention towards the quality assurance of such systems. Despite the advances in software testing, MLSs bring novel and unprecedented challenges, since their behaviour is defined jointly by the code that implements them and the data used for training them. Objective: To identify the existing solutions for functional testing of MLSs, and classify them from three different perspectives: (1) the context of the problem they address, (2) their features, and (3) their empirical evaluation. To report demographic information about the ongoing research. To identify open challenges for future research. Method: We conducted a systematic mapping study about testing techniques for MLSs driven by 33 research questions. We followed existing guidelines when defining our research protocol so as to increase the repeatability and reliability of our results. Results: We identified 70 relevant primary studies, mostly published in the last years. We identified 11 problems addressed in the literature. We investigated multiple aspects of the testing approaches, such as the used/proposed adequacy criteria, the algorithms for test input generation, and the test oracles. Conclusions: The most active research areas in MLS testing address automated scenario/ input generation and test oracle creation. MLS testing is a rapidly growing and developing research area, with many open challenges, such as the generation of realistic inputs and the definition of reliable evaluation metrics and benchmarks.
-
Language
-
-
Classification
-
Computer science and technology
-
License
-
License undefined
-
Identifiers
-
-
Persistent URL
-
https://n2t.net/ark:/12658/srd1319375
Statistics
Document views: 103
File downloads: