Assessing the Effect of Data Transformations on Test Suite Compilation

Panagiotis Stratis, Vanya Yaneva, Ajitha Rajan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Background. The requirements and responsibilities assumed by software has increasingly rendered it to be large and complex. Testing to ensure that software meets all its requirements and is free from failures is a difficult and time-consuming task that necessitates the use of large test suites, containing many tests. Large test suites result in a corresponding increase in the size of the test code that sets up, exercises and verifies the tests. Time needed to compile and optimise the test code becomes prohibitive for large test code sizes. 
Aims. In this paper we demonstrate for the first time optimisations to speedup compilation of test code. Reducing the compilation time of test code for large and complex systems will allow additional tests to be compiled and executed, while also enabling more frequent and rigorous testing. Methods. We propose transformations that reduce the number of instructions in the test code, which in turn reduces compilation time. Using two well known compilers, GCC and Clang, we conduct empirical  evaluations using subject programs from industry standard benchmarks and an industry provided program. We evaluate compilation speedup, execution time, scalability and correctness of the proposed test code transformation. 
Results. Our approach resulted in significant compilation speedups in the range of 1.3× to 69×. Execution of the test code was just as fast with our transformation when compared to the original while also preserving correctness of execution. Finally, our experiments show that the gains in compilation time allow significantly more tests to be included in a single binary, improving scalability of test code compilation.
Conclusions. The proposed transformation results in faster test code compilation for all the programs in our experiment, with more significant speedups for larger case studies and larger numbers of tests. As systems get more complex requiring frequent and extensive testing, we believe our approach provides a safe and efficient means of compiling test code.
Original languageEnglish
Title of host publicationProceedings of 12th International Symposium on Empirical Software Engineering and Measurement (ESEM’18)
Place of PublicationOulu, Finland
Number of pages10
ISBN (Electronic)978-1-4503-5823-1
Publication statusPublished - 11 Oct 2018
Event12th International Symposium on Empirical Software Engineering and Measurement - Oulu, Finland
Duration: 11 Oct 201812 Oct 2018


Conference12th International Symposium on Empirical Software Engineering and Measurement
Abbreviated titleESEM2018
Internet address


Dive into the research topics of 'Assessing the Effect of Data Transformations on Test Suite Compilation'. Together they form a unique fingerprint.

Cite this