AI and the quest for diversity and inclusion: a systematic literature review

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

The pervasive presence and wide-ranging variety of artificial intelligence (AI) systems underscore the necessity for inclusivity and diversity in their design and implementation, to effectively address critical issues of fairness, trust, bias, and transparency. However, diversity and inclusion (D&I) considerations are significantly neglected in AI systems design, development, and deployment. Ignoring D&I in AI systems can cause digital redlining, discrimination, and algorithmic oppression, leading to AI systems being perceived as untrustworthy and unfair. Therefore, we conducted a systematic literature review (SLR) to identify the challenges and their corresponding solutions (guidelines/ strategies/ approaches/ practices) about D&I in AI and about the applications of AI for D&I practices. Through a rigorous search and selection, 48 relevant academic papers published from 2017 to 2022 were identified. By applying open coding on the extracted data from the selected papers, we identified 55 unique challenges and 33 unique solutions in addressing D&I in AI. We also identified 24 unique challenges and 23 unique solutions for enhancing D&I practices by AI. The result of our analysis and synthesis of the selected studies contributes to a deeper understanding of diversity and inclusion issues and considerations in the design, development and deployment of the AI ecosystem. The findings would play an important role in enhancing awareness and attracting the attention of researchers and practitioners in their quest to embed D&I principles and practices in future AI systems. This study also identifies important gaps in the research literature that will inspire future direction for researchers.

Related collections

Most cited references 40

Record: found
Abstract: found
Article: not found

Dissecting racial bias in an algorithm used to manage the health of populations

Ziad Obermeyer, Brian Powers, Christine Vogeli … (2019)

Health systems rely on commercial prediction algorithms to identify and help patients with complex health needs. We show that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias: At a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses. Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%. The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients. Thus, despite health care cost appearing to be an effective proxy for health by some measures of predictive accuracy, large racial biases arise. We suggest that the choice of convenient, seemingly effective proxies for ground truth can be an important source of algorithmic bias in many contexts.

0 comments Cited 954 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

A Survey on Bias and Fairness in Machine Learning

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena … (2021)

With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.