September 18, 2020 / By Jenny Riley & Jess Dart
Over the past few years, we’ve noticed a trend towards “designers” (people skilled in product design, industrial design and human-centered design) taking a greater role in organisations working on social and environmental innovations. This is good news for change makers, as designers bring a great toolset for achieving change in our sectors, including the capacity to better test and ensure that products and services are effective and engage users. But it’s also caused a bit of heat in some kitchens, as evaluators and designers clash over whose role it is to monitor, test and evaluate the initiatives.
While working with The Australian Centre for Social Innovation (TACSI), Jess Dart has been known to refer to this dynamic between designers and evaluators as “the cook and the chef”. The designers we’ve worked with tend to be highly pragmatic, “get in there and give stuff a go”-types who are not afraid to prototype with what they have, nor afraid to fail fast. Evaluators, on the other hand, can be a bit more classical in their approaches, deliberate and focused on method and the final assessment.
So how can and do these roles work together? Is it like Maggie Beer and Simon Bryant, the original cook and chef, who share a love of cooking and a collaborative approach, with Simon prepping Maggie’s herbs and Maggie taking Simon’s corn tamales out of the oven? Or is it more like My Kitchen Rules, with every cook following their own recipe, and paring knives at the ready?
We think there are four ways this dynamic can play out:
This is the original Maggie and Simon model, where there is great synergy and an organisational approach. The designer and the evaluator each come with their own tools, experience, and knowledge, but they work collaboratively on monitoring, measuring, testing, and evaluating product design and impact. Evaluators bring their robust theories of change and understanding of cause and effect and devotion to outcomes, while designers bring their journey maps and their devotion to the user experience.
In this scenario, testing and monitoring become one and there is joint accountability. Designers and evaluators bring design and evaluative thinking at every stage in the journey. So evaluation contributes to product development all the way through from the initial testing phase to scaling stage.
The Cook and the Chef work together as one team. This is the 3-star Michelin experience that best practice points us to – but like Michelin stars, they’re expensive and few and far between.
In this model, evaluators take an oversight role in the design process, a bit like the judges in Masterchef. Designers are doing the work, making the product, and the evaluators are looking across this process and making judgements about the overall design process.
In this model there is a focus on process evaluation. The emphasis is on accountability and quality, and it’s judgemental – how well has the process gone? How well were users engaged, to what extent was the model backed by evidence, and at the end of the day, was it worth it?
So in this kitchen, the chef checks on the cook, tasting the food and commenting on the outputs of the process as a kind of quality assurance process.
The cooks are loose in the kitchen! In this scenario, designers take the lead and outline the approach for evaluators to work behind them on product design and impact evaluation.
Learning is fast and there are quick iterations – and perhaps there are some spills and pots boiling over, but lots of recipes are created and tested on customers. Evaluators support the designers – but only when they indicate a desire for support and in some scenarios, are at the beck and call of designers.
In this model, evaluators set up questions, provide insights, facilitate learning in a more developmental evaluation way of working, but only when called on, for example, when reports or quality reviews are required. The emphasis is on learning and impact as well as the other domains.
In this scenario – one we’re seeing more and more of – there is a clear demarcation of roles. Here the designers and evaluators have delineated responsibilities in relation to measurement and evaluation.
The designers take the lead in undertaking measuring and evaluation activities in the prototype, pilot, and early release phases and then they hand over the product to the evaluators for evaluation. Evaluators step in at the stage of general release and not before. Or in kitchen speak, the cooks set the menu and prepare the meal, and the sous chefs step in to add their final, specialist touches.
The emphasis is on reach, effectiveness and impact domains.
It’s not always going to be as clear cut as these four kitchen scenarios suggest, but when dealing with energetic cooks and seasoned chefs, we’ve found it useful as a model for working out who does what and when.
We’d love to hear from you – does one of these sound like the kitchen you’re working in? What ingredients are we missing?
132B Gwynne St, Cremorne 3121 VIC
P (+61) 3 9425 7777 | info@clearhorizon.com.au
Clear Horizon acknowledges Aboriginal and Torres Strait Islander people as the Traditional Custodians of the land and acknowledges and pays respect to their Elders, past and present.
Hands Up Mallee – The community taking children’s wellbeing into their...