What’s the problem?
AI can benefit people’s work and everyday lives through decision support systems, finding information and AI-generated text or images. Yet, high-profile failures, such as predictive AI models that cast aside job applications by women or generative AI helping lawyers write judicial arguments that ‘hallucinate’ non-existing court cases, are causing serious concerns. A significant barrier to reaping the benefits of predictive and generative AI is their unassessed potential for harms: an AI that is of poor quality in terms of truthfulness, transparency, information security, bias and robustness which may cause unfair outcomes, treatment and wider harms. Hence, AI auditing has become necessary, in line with existing and impending regulation that calls for regular assessments (e.g. US President Biden’s Executive Order and the EU AI Act).
Yet, AI auditing has been haphazard, unsystematic, without common frameworks or standards and left solely in the hands of experts. Our project aims to fix this fundamental challenge through the novel concept of participatory AI auditing, where a diverse set of stakeholders, such as domain experts, regulators, end-users and decision subjects, are empowered to undertake audits of predictive and generative AI, both individually and collectively. To this end, our project will produce workbenches that support these stakeholders in auditing AI, and embed these audits in methodologies that define how, when and who carries out the audits. We will train stakeholders in carrying out audits and work towards a certification framework for AI solutions that use participatory AI auditing.
Our project is situated against a background of research and practice in AI safety testing and AI evaluations and audits. Some tools (e.g. Google’s What-If tool, or IBM’s AI Fairness 360 system) now exist to help predictive AI solutions to be audited by AI experts in terms of accuracy and bias, using fairness measures. Some auditing methodologies for generative AI,
such as red teaming, have been taken up by industry.