Our Vision
Our project will investigate how to enable individual and collective participatory auditing of current and future AI technologies so that diverse stakeholders beyond AI experts can be involved in auditing their harms. Our research will systematically investigate stakeholders’ needs for and notions of fairness and harms to create auditing workbenches comprising novel user interfaces, algorithms and privacy-preserving mechanisms that help stakeholders to perform audits whilst guarding against unintended negative effects or abuse by malicious actors (i.e. biased auditors). We will create participatory auditing methodologies which reflect, anticipate and inform regulatory frameworks, specifying how to embed participatory auditing in the AI development lifecycle using the workbenches we have developed. We will develop and implement training for stakeholders in participatory auditing to embed our project outputs in practice. We will work towards a certification framework for AI solutions, thus ensuring that AI is safe and trustworthy.
The example use cases we will focus on in detail
For predictive AI, we tackle two distinct use cases, Health and Media Content. In Health, we will work initially on two datasets with two different sets of stakeholders; this will enable us to investigate how our findings generalise across contexts in the same domain. Scottish Patients at Risk of Readmission and Admission (SPARRA), developed by NHS NSS and Public Health Scotland (both involved in this project), helps healthcare professionals by predicting a person’s likelihood of being admitted to hospital as an emergency in-patient within the next year. This is calculated monthly for approximately 80% of all people living in Scotland. Previous investigations by AI experts have already revealed bias in SPARRA but participatory auditing is hampered because the detailed dataset is not available publicly. Stakeholders for this context are healthcare professionals, NHS decision makers and patients. The second dataset, the School Attachment Monitor Corpus contains videos of children undergoing the Manchester Child Attachment Story Task. It aims to help with the identification of children with insecure attachment by using a variety of social signal processing methods. This dataset suffers from bias, privacy issues and a lack of agreement about ground truth. Stakeholders include psychologists, social workers, educators, and parents of children undergoing assessment. In the Media Content use case, we will address fairness in a search engine setting in collaboration with Istella. An unfair search engine can result in amplified misinformation and biased perspectives being propagated through society, as well as discriminating against groups that are marginalised in the search results leading to a loss of real-world opportunities and a negative economic and social impact. Potential stakeholders in this use case are search engine developers and users. We also will work on the publicly available Jigsaw dataset, which has been used to detect hate speech in social media and suffers from bias in hate speech classification. Stakeholders will include social media moderators as well as social media users.
For generative AI, we will work on two use cases. For Cultural Heritage, we will employ datasets provided by project partners National Library of Scotland, Museum Data Service, and the David Livingstone Birthplace, involving collections of historical material including newspapers, books, manuscripts, records, and poem collections. Generative AI allows users to ask questions about the content, offers new interpretations of the material, annotates the material, and offers speculative explanations of phenomena. The risks of generative AI in this situation include misrepresentations, such as hallucinations, historical bias arising from the data, or lack of consideration of the context in which historical events have occurred. Stakeholders include collections’ curators as well as their users. For Collaborative Content Generation, project partners Wikimedia Foundation and Wikidata will support our research into generative AI used to write articles, focusing on under-resourced languages. There are more than 300 language editions of Wikipedia, but their number of articles and editors varies greatly. For example, while Arabic is the fifth most spoken language by number of speakers, Arabic Wikipedia has only 10% of the articles compared to the English language version, created and maintained by fewer than 4k editors. Given Wikipedia’s role as a trusted information source, this can entrench existing inequalities in accessing and sharing knowledge, hamper cultural diversity and heritage efforts, create unfair erasure of marginalised voices and representation, and contribute to the spread of misinformation. More perniciously, foundational AI models also use Wikimedia articles as training data, leading to a potential feedback loop of harms. We will focus on articles about health, linking to use case 1. Stakeholders will include article editors, and well as users of article and information written by AI, including fact checkers, journalists and researchers, supported by project partners Full Fact and the Open Data Institute.