Constellation, le dépôt institutionnel de l'Université du Québec à Chicoutimi

Résumé

Context: Between 2015 and 2020, the annual installations of industrial robotic systems increased by 9% each year. This rise in industrial robots has prompted researchers and practitioners to implement software engineering approaches within robotic domains to enhance quality and address failures. However, challenges and gaps related to software testing for robotic systems still exist. Goal: We proposed and evaluated a software testing approach called Automated Acceptance testing for Industrial Robotic Systems (AAT4IRS), which aims improve the effectiveness of testing by enhancing fault detection and test robustness. Additionally, we propose an evaluation guideline for applying mutation testing to industrial robotic systems that incorporate unpredictable actions, thereby increasing trust in simulation testing. Method: First, we conducted a systematic analysis of software engineering and software quality practices applied in robotic systems. Since our research focuses on the intersection of software testing and industrial robotic systems (IRS), this preliminary analysis provided valuable insights into how roboticists integrate software engineering techniques in their work. Secondly, we defined and validated an automated acceptance testing approach that enhances the effectiveness of testing by improving fault detection and test robustness. This proposed approach utilizes Behavior Driven Development (BDD) templates for acceptance testing. Lastly, we developed and evaluated a guideline for applying mutation testing to IRS, which includes the unpredictable actions typically observed in these systems. This incorporation of domain-specific details aims to improve trust in simulation testing. Results: The systematic mapping shows me that there is a consensus by the research community that software engineering techniques and approaches can improve the quality of robot software. Moreover, the majority of the studies focus on three areas that performed activities inherent to robotics software development (software design, construction and models and methods). However, only 18% of the studies performed activities related to testing and quality. To evaluate the effectiveness of our software testing approach, we tested the generated test suites against mutants created from the original code. The test suites produced through the use of Automated Acceptance Testing for Industrial Robotic Systems (AAT4IRS) achieved an effectiveness score of 90%. In addition, the results observed in our experiment suggest that AAT4IRS has some benefits such as: (i) reduction in the effort to discover faults in IRS; (ii) improvement in traceability; (iii) better coverage of regression test; and (iv) improvement of coordination, collaboration, and documentation in robotic systems development teams. All these benefits needs to be evaluated in future work, but as they are known benefits by applying software testing in conventional system, we strongly believe that they also can also benefit robotic systems.