6 questions evaluators should be asking ourselves if we’re serious about change
Reflections on the AES22 International Evaluation Conference
This month, evaluators and change makers from across the country came together for the AES22 International Evaluation Conference – our first face-to-face gathering in three years.
The theme of the conference was weaving evaluation into the whole, looking at:
- how evaluation can involve and impact everyone: how evaluation weaves into programs and policies at community, state, and national levels.
- whose voices and values matter in evaluation; exploring inclusive approaches embrace perspectives of First Nations people and diverse communities.
- innovation in evaluation practice; how evaluation has adapted amid acute and chronic challenges.
The speakers and sessions I saw were insightful and inspiring – and as usual, left me with much to reflect on. I think anyone working in this space can feel the shifts occurring beneath our feet: if we truly want to use our evaluative and impact measurement skills for positive impact, we need to step-up. So here’s what I’m asking myself – and what I challenge you to consider in your daily work.
1. How can evaluation directly contribute to creating a better world?
Evaluation can and should be about so much more than accountability to a funder. And as societal challenges pile up, we need to shift from the ‘neutral’ role we often play, to an active one in achieving change. This might be thinking about how we might include questions about equity or climate change into every evaluation, even when they’re not in scope. Or it could see us treating evaluation as more of an intervention where we embed learning and adaptation into social change initiatives from the get-go, much like we do when using a Developmental Evaluation approach.
2. As evaluators, how might we balance not over or under reaching in our work?
Amy Gullickson spoke bravely about what we as individuals bring to evaluation. How our own anxiety can lead to overreach, for example, when we end up doing the client’s work ourselves, or underreach, when we tap-out or become overly passive. As someone personally guilty of overreaching, I found her suggestions of focussing on self, staying connected and curious in order to free up energy to explore new directions to be thought-provoking. I’ll definitely be considering how we might think about our own purpose, practice, boundaries and self-care as evaluators.
3. How can we be better allies for First Nation evaluators and organisations?
Nan Wehipeihana, a legend in the evaluation world, presented on Evaluation in Pursuit of Indigenous Health Equity. She started off with a clear theoretical explanation of evaluation through an equity lens – ‘one size fits one, rather than one size fits all’. She also challenged us – evaluation is not ‘value free’, and our role is to be stewards of public good and be brave in calling out inequity (linking to point 1 in being more ‘active’ in our role as evaluators). She also reinforced that first nations evaluations should be led by first nations evaluators.
4. How might we center people with lived experience into every evaluation?
It’s a sad truth, but sometimes it’s the very people’s whose experiences are being evaluated who have the least say in an evaluation. We might not hear about the true impact of an initiative because we’re not asking the right questions – because we haven’t listened to the right people when designing our evaluation. We need to better consider how we bring the experiences of those being evaluated into focus. We also need to consider how can best capture diverse views. While a number of sessions touched on this, Lauren Siegmman had some points about how evaluation as a profession could invest more in seeking the perspectives of people and organisations representing ‘the evaluated’, and especially those who have had had negative experiences. Often in evaluations, those with positive experiences are more likely to be centred. We also need to get better at capturing the voices of those who experienced negative or unintended consequences, as my colleagues Niketa Kulkarni, Nikki Bartlett and Sarah Leslie discussed.
5. How can we better equip change-makers to create transformational change in complex settings?
Again linking to evaluators becoming more active, how might we better integrate evaluating thinking with impact design to help “de-risk” social innovation? And support innovation and evaluation across complex systems and community settings? The conference showcased several great examples of evaluation being baked right into the start of an initiative with sequenced developmental, formative and summative phases. I even heard about evaluators re-framing their whole evaluation approach mid-stream to incorporate systems thinking, as well as inspiring examples of rapid evaluation. As evaluators, we’ve got much more to offer than an end-of-project report – we need to consider how and where we can help create the greatest impact.
6. As a profession, how can we get ready for the digital transformation that’s coming?
Did you know that the Internet of Things, Natural Language Processing and Blockchain technology all can have valuable uses when it comes to evaluation? I certainly didn’t, and was grateful to my colleague Ethel Karskens for helping demystify how current and future technology can offer greater efficiencies and insights. Automation, connectivity and machine learning are being used across multiple industries, and now increasingly in evaluation. Those not keeping up will find it increasingly difficult to catch up, so how can we ensure we don’t get left behind?
