Reinforcement learning for optimizing regression testing

Abstract

The agile manifesto emerged in 2001, introducing principles that emphasize flexibility, customer satisfaction, and continuous improvement. Continuous Integration (CI) supports these Agile objectives by ensuring that code changes are automatically tested and merged frequently, leading to early detection of issues and allowing teams to adapt quickly. Combining CI with regression testing is a widely recognized as an effective approach in software development, ensuring that new changes do not adversely affect existing functionality. However, regression testing can be challenging due to the time and resources required to test the entire system at every iteration. To address these challenges, one alternative is Test Case Prioritization (TCP). This involves strategically ordering test cases to detect regression faults earlier.

The primary objective of this thesis is to streamline the CI build cycle by improving the fault detection rate. This enhancement will enable software developers to integrate their changes more rapidly and continuously. This thesis adopts the systematic methodology of the Design Science Research (DSR) and was conducted in three phases. First, we reviewed relevant literature to identify opportunities for further research and examine the existing knowledge base. The second phase involved developing a history-based TCP method using Reinforcement Learning (RL), which was subjected to multiple iterations of improvement and evaluation. In the final stage, the results from the initial two stages were synthesized to draw conclusions and identify practical implications.

In this thesis, TCP is modeled as a ranking problem addressed by the RL agent.This agent iteratively adjusts its actions based on feedback (rewards) from its environment, learning to rank test cases adaptively in an optimal order. For RL model development, we utilized the stable-baselines3 framework, specifically employing algorithms like Advantage Actor-Critic (A2C), Proximal Policy Optimization (PPO), and Deep Q-Networks (DQN). The performance of each algorithm was assessed using the Average Percentage of Fault Detected (APFD) metric. The method was trained on 963 CI builds, encompassing 44,000 test cases extracted from the Nokia CI repository.

The primary contribution of this thesis is the is the creation and assessment of an RL-based TCP method that is accurate in prioritizing test cases and adaptable to changes in CI environment. The experimental results underscore the efficiency of the A2C algorithm, which achieved a mean APFD score of 0.78 across 963 CI cycles, with average testing and training times per cycle of 322 ms and 54129 ms, respectively. Based on these findings, it's evident that RL is a promising approach in software testing, particularly effective in managing the complexities of dynamic testing environment such as the CI environment.