Standard-compliant testing for safety-related automotive software

This thesis proposes a new testing method called CTMCONTROL, which enhances the popular Classification Tree Method (CTM) by incorporating Modified Condition/Decision Coverage (MC/DC) capabilities. This brings the CTM closer to the quality assurance requirements of the latest standards that regulate automotive software. CTMCONTROL has a particular focus on the specification-based control-logic of the system under test (SUT). It incorporates MC/DC coverage of the logical expressions guarding the transitions of a system represented via Statecharts and it allows testing of the dependencies among the input values and the control logic of the SUT. The experimental results reported in this thesis point to the fact that CTMCONTROL can capture a group of errors in the control-logic of the SUT, which would not be captured via the classic CTM approach. The additional group of errors captured by CTMCONTROL, includes errors which can cause unwanted activations or unwanted feature interactions in a system. CTMCONTROL has been prototypically implemented in the MATLAB/Simulink/Stateflow environment. This thesis also proposes the Formalization of a Reactive Testing Process (FRTP) which builds a pathway between reactive testing and the field of formal testing. The FRTP defines a number of ordered steps to be followed in a reactive testing process and a flow of logic on how to move between them. The steps serve as methodological pointers to help with the integration of formality in the automotive software testing process. The FRTP includes a new Taxonomy of Reactive Testing (TRT). The TRT helps the test designer to elucidate the reactivity dimensions which are of interest for a specific testing process. To facilitate the use of formalization in industry, examples specified in Z notation are given for each of the dimensions of the TRT. The examples are meant as informational models, which can guide the “typical automotive software tester” in the formalization of test reactivity. Also, changes are proposed to the latest model-based testing taxonomy, via incorporating test reactivity as one of the test-selection criteria. This accounts for those reactive test processes where test generation is performed on the fly. All of the contributions of this research are tailored to the field of safety-related automotive software.