Reflections from the American Evaluation Association’s Annual Conference 2024
Written by Zoe Enticott
Two weeks ago, I had the pleasure of attending the American Evaluation Association’s (AEA) annual conference in Portland, Oregon. Kate McKegg and I attended to present on our three-year Developmental Evaluation of the Stewardship of Aged Care initiative, which we are undertaking with an outstanding team in the Commonwealth Department of Health and Aged Care.
The conference theme, “Amplifying and Empowering Voices in Evaluation“, resonated deeply with me. Evaluators at all career stages being encouraged to bring forth new ideas and creative approaches to our field. With nearly 3,000 people in attendance, the conference centre buzzed with the energy of shared passion, thoughtful debate, and a willingness to challenge the status quo.
The opening plenary was especially moving. Dr. Ricardo Millet, renowned for his dedication to fostering diversity within evaluation, was honored. Surprisingly to me, he was honoured not for technical achievements, but for his relational and mentoring skills. Witnessing him being celebrated for his empathy and leadership reminded me that evaluation is as much about human connection and representation as it is about our methods and methodologies.
Key Takeaway #1: The AI Revolution is Here and It’s for All of Us
A prominent theme of the conference was the rise of artificial intelligence in our field. There’s no doubt that AI has arrived. These days, anyone connected to the world wide web has easy access to models trained on enormous datasets that can respond in real time, answer complex questions, write creatively, and even support us with complex data analysis. Notably, conversations around AI’s potential weren’t limited to “digital experts” or tech specialists. Evaluators and systems-thinkers were all being encouraged to experiment, as we all explore this technology’s applications together.
One fascinating session by systems innovator Jewlya Lynn proposed a unique angle: AI as another
“stakeholder at the table.”
Lynn demonstrated three ways AI can actively support systems innovation beyond data processing.
She described how her team are using AI to:
- prepare for facilitation, by scanning the literature and summarising different perspectives on key issues we are facing in the poly-crisis,
- act as a thought partner during real-time facilitation, where a group throw to a well-trained AI to test their assumptions or gather new perspectives, and
- to assist in describing and unpacking complex systems.
By engaging AI as if it were a collaborator, Lynn and her team can ask it questions and draw on a wide array of perspectives, provided they use carefully designed prompts to minimise bias.
This approach made me rethink the role AI might play, not simply as an efficiency tool supplementing human labour, but as a tool that enhances our ability to make sense of complexity and systems. Critical to application in this way is our ability to identify and reduce bias, which Lynn had strategies for.
Key Takeaway #2: Positionality is Key – “Know Thyself”
Another recurring theme was positionality—the understanding that we all bring our identities, beliefs, and worldviews into our work and should reflect upon our position in relation to other social identities. In their panel discussion, editors of a forthcoming book: Theories Bridging Ethnography and Evaluation described how they use social inquiry, reflexivity, authenticity, and reciprocity to practice in underrepresented contexts and with historically marginalised communities. I was reminded of the value of ongoing and critical self-reflection to make conscious the ways in which our biases, and the roles we play shape our evaluation processes and findings.
Jewlya Lynn and Julia Coffman explored a related concept in their engaging workshop on mental models about systems change. They contrasted two models common in philanthropic systems-change work: Systems Dynamics, which assumes a linear and predictable pathway to change, and therefore sets targets and causal pathways to achieve outcomes, and Systems Emergence, which fundamentally sees change as non-linear, and therefore encourages a higher degree of experimentation and adaptability. Both are powerful and both can be impactful, but left uninterrogated, differences between these models can create misalignment and conflict, particularly in complex environments.
While Lynn and Coffman did not favour one mental model over another, there was a sense that certain system features are better suited to certain mental models. For example, when dealing with a problem that is not well studied or understood, our ability to apply a Systems Dynamics model, where we pre-plan our actions and intended outcomes may prove detrimental, as our assumptions crumble in the face of emergent and unpredictable outcomes.
This tension is one I see play out in my work as a consultant, where I encounter both mental models. Increasingly I’m noticing some of the pitfalls of using rigid, pre-defined methods and methodologies to address problems that exist within more emergent, complex environments, where change is difficult to predict and plan for. I’m grateful to work with clients who understand the value of adaptability when operating in complex environments but the tensions navigating different mental models still exist.
Key Takeaway #3: Embrace New Epistemologies – “Expand Thyself”
The closing plenary stirred another powerful conversation, this time about the dominance of Western worldviews in evaluation. Professor Bagele Chilisa called for a greater embrace of Indigenous and post-colonial perspectives, highlighting the value of social, spiritual, and relational knowledge that Indigenous paradigms bring to our work. Indigenous ways of knowing emphasise interconnectedness, respect for relationships, and holistic approaches –- elements that the evaluation field, and perhaps our world, urgently need.
Afterward, I reflected on these points with a colleague, noting the profound wisdom in Indigenous epistemologies. As white practitioners, we felt compelled to learn more about these ways of working, and to continue de-colonising our practices through self, and group reflection. I left the conference with a renewed commitment to not only expand my technical skills but also deepen my understanding of the diverse perspectives that can inform and strengthen our work.
Moving Forward: Amplifying and Empowering New Voices in Evaluation
The AEA conference left me inspired, challenged, and ready to approach my work with fresh insights. As evaluators, we are not just data collectors or analysts; we are bridge-builders and connectors, tasked with bringing together diverse voices to create more equitable outcomes. Whether it’s using AI as a collaborative tool, reflecting on our own positionality, or embracing Indigenous epistemologies, I’m excited to bring these learnings into my practice and, hopefully, inspire others to do the same.
Mostly, I feel privileged to be surrounded by such intelligent, caring, and thoughtful people willing to share what they have learned to continue building our field. In future, I look forward to sharing some insights from our developmental evaluation within a large-scale system reform.
