59 - The Next Big Thing in Evaluation is AI, But for Whom?

Abstract Information: Artificial intelligence (AI) applications, particularly those involving machine learning and natural language processing, have emerged as the “next big thing” in evaluation. However, before we grasp yet another shiny object, it is essential that we first consider the ethics and duties of our profession. We are responsible for ensuring that the voices of our funders, their awardees, and the diverse communities served are included as we incorporate innovative technological methods into practice. In this session, we apply Everett Rodgers (1962) diffusion of innovation theory to explore the inclusivity of our AI modeling practices. According to Rogers, an innovation must have five characteristics to spread: (1) high relative advantage, (2) testability, (3) observability, (4) compatibility, and (5) low complexity. We then show how we can use these characteristics as heuristics that aid in identifying how to include diverse voices in the products and processes of AI modeling. Finally, we discuss how computational simulation modeling can be used in conjunction with AI modeling to help specify and amplify the voices of stakeholders. Building simulation models requires the explication of the system’s boundaries to be modeled, an understanding of the underlying interdependencies, and the inclusion of multiple perspectives and diverse voices to capture the system’s complexities. The product of the conversation will be a list of initial steps that might elevate inclusivity as a sixth characteristic driving the diffusion of innovation.