Scientific Program

Context

AI-based decision-making systems have become ubiquitous and an integral part of various aspects of our modern society. With their widespread development, ensuring the privacy and fairness of AI systems outcomes has emerged as a critical challenge and an ethical responsibility.

Privacy auditing of ML systems

Central to our scientific context is the foundational concept of Differential Privacy (DP), which has gained widespread adoption by Big tech companies as well as by government agencies, making it the leading standard of modern privacy preservation. Nonetheless, fine-tuning the appropriate noise level can pose a complex challenge as the DP definition considers a worst-case scenario. In addition, implementing DP mechanisms for ML involves intricate aspects such as micro-batching, sensitivity analysis, and privacy accounting, in which errors can easily occur.

To mitigate the risk of unintentionally exposing sensitive data, our research strongly advocates for data-driven approaches to auditing DP mechanisms. More precisely, we believe that the privacy auditing tools developed will enable us to uncover potential vulnerabilities or flaws in practical implementations of DP mechanisms that might not be apparent through theoretical analysis alone. Furthermore, empirical estimation of privacy loss through realistic attacker models will assist practitioners in making informed decisions for the implications associated with the choice of specific privacy parameter values.

Fairness auditing of ML systems

Another critical concept to our context is fairness in machine learning, which is paramount in our digital age in which algorithms impact crucial aspects of our lives, from job opportunities to financial outcomes. Thus, ensuring that these algorithms are fair and unbiased is essential to prevent discrimination and societal inequalities. To achieve this issue, we propose in this project to take a multidisciplinary approach, involving a legal investigation concerning the consequences that such development of AI systems might entail for data subjects, in particular when used for automated decision-making processes.

There have been many efforts to develop methods and metrics to evaluate and promote fairness in ML due to unequal treatment of individuals or groups based on factors such as race, gender, or socio-economic status. However, despite the recent proliferation of various bias metrics and fairness definitions, a consensus on which metric is best adapted to a specific context is currently lacking. Additionally, there is a scarcity of accessible resources to effectively put these metrics and definitions into practice. Consequently, even with growing awareness, the practice of auditing for bias and ensuring fairness during the development and deployment of AI systems is far from being a widely established standard.

Objectives

O1. Privacy Audits: 

This objective aims to assess the privacy loss of AI systems under realistic attackers, particularly those built with DP guarantees. One main challenge of this task will be to design attacks on the DP mechanisms by considering different threat models (i.e., central DP, local DP, shuffle models, …) and different background knowledge of the adversary. First, we will rely on Monte Carlo methods to run a statistical hypothesis test and derive the privacy loss by calculating the confidence intervals of the corresponding Bernoulli random variables from n independent trials of the mechanism. In this context, another challenge for DP auditing will be designing and developing efficient audit tools that do not require running thousands of trials n but that, at the same time, achieve a tight estimate of the privacy loss.

O2. Fairness Audits:

The second objective revolves around auditing the fairness of AI systems. While the design of fair algorithms is a crucial step in addressing societal biases, ensuring their fairness in practice is equally vital.
Our motivation lies in the need to bridge this gap between theory and practice, conducting fairness audits that inspect algorithmic decisions for real-world scenarios. For instance, many fairness metrics are at odd and, thus, we aim to design a generic audit tool covering different fairness metrics. Moreover, despite rigorous theory efforts, AI systems can still produce biased outcomes for unseen data (testing data), often due to nuances in data or unexpected interactions. We aim to tackle this particularity by identifying and designing robust fairness mechanisms that work in practice. We aim to develop practical fairness auditing methodology/tools that will serve as a critical catalyst in addressing these challenges. 

O3. Combined Privacy and Fairness Audits:

As for the third objective, we recognize the importance of addressing privacy and fairness in a combined manner. For instance, recent works have shown that applying privacy-preserving mechanisms such as DP can significantly harm sub-groups of the population. In addition, training ML models to be fair usually requires access to private information (e.g., the sensitive attribute) thus also raising privacy issues. Our motivation is to develop audit methodologies, tools, and systems that consider the interplay between these two critical dimensions to assess AI systems comprehensively. In addition, we want to ensure that our audit techniques are robust to an adversarial situation in which the provider of the AI system decisions might be tempted to perform ethics washing, which can be defined as promoting the false impression that a particular AI system respects some ethical values while it might not be the case.

Summary of Activities

• 2024-02 to 2024-03: Visit of Guillaume Gagnon (Ph.D. student, UQAM) to INSA-Lyon to work on privacy implications in IoT devices.