Constellation, le dépôt institutionnel de l'Université du Québec à Chicoutimi

Résumé

Artificial Intelligence (AI) has significantly transformed various aspects of our lives, yet concerns have arisen regarding security, trustworthy, reliability, and effectiveness,particularly in safety-critical systems. These systems can be biased, complex, and uncertain, leading to potential human or financial loss. An AI system should be robust, resilient, and transparent to ensure safety. The increasing adoption of these AI systems in critical domains such as healthcare, finance, and transportation has raised significant concerns about their decision’s reliability, robustness, and trustworthiness. Various safety standards and principles are presented for critical systems. Current research focuses on improving AI systems, but little research has been done on ensuring safe AI. To develop and maintain safe AI, potential risks (including bias, data security, and susceptibility to external threats) must be identified, and procedures must be established to prevent and reduce them. Fairness, robustness, reliability, and AI alignment are the key criteria for Safe AI systems.

To ensure fairness and robustness, the development of AI models requires sufficient training data in terms of quantity and quality to make meaningful predictions. Current research focuses on training large models to make general decisions. However, training these models requires a lot of computational capacity and training data. To resolve the limited training data problem, researchers employ collaborative learning (aka Federated Learning), which collaborates with several smaller groups to pool their computational resources and local data and train a global model that benefits all participants. However, their reliability and safety are still a serious concern, especially in critical domains where errors or biases can have severe consequences.

The research begins with a conceptual framework of Safe AI, demystifying its principles and clarifying its role in fostering secure, robust, trustworthy, and ethically aligned AI systems.Within this context, collaborative learning emerges as a promising approach for safety-critical systems, but it faces two fundamental problems : (i) data heterogeneity across subgroups, which undermines fairness and robustness, and (ii) vulnerability to adversarial threats, particularly data poisoning, which compromises security and reliability.

To address these challenges, two novel contributions are presented. First, FedCIM introduces a mechanism to mitigate the effects of heterogeneous data distributions in collaborative learning, thereby enhancing fairness among participants and improving model robustness across diverse datasets. Second, RBFL proposes a reputation-based defence strategy that secures collaborative learning against data poisoning attacks, ensuring resilience and trustworthiness in training environments. Together, these contributions advance the state of the art in collaborative learning by embedding fairness, robustness, and security into its design. By integrating theoretical insights with practical solutions, this thesis demonstrates how the vision of Safe AI can be realized for safety-critical systems..