Impact assessment
Impact assessment determines if projects are funded or can proceed or are canceled. These assessments also help to quantify and qualify what occurs in a change project.
An impact assessment is based on past evidence (what happened in similar cases) and experience, yet also allows for speculation on the good or bad effects that a project can have. In addition, there is judgement made about comparative impacts and which dominates in any particular case.
Many sectors have their own methods to make these assessments and use specific indicators to measure and validate the impact. For example, an Environmental Impact Study will be used to assess if a mining project will, on balance, do harm to the environment or limited, acceptable harm. The Impact assessment of an International Development engagement can determine if a funder is prepared to invest money in the project. Or in the case of academic research, Impact is used as a way to measure the potential value of the research, hence its chances of being funded.
In the three cases — Energybank, Homeful and Plastics — the impacts of change on their communities and systems, will be different for each. In this course it is not possible to speculate on the potential impact for all aspects of the implementation model you have constructed. However, it is useful to use an Impact Assessment as a way to review your work and identify, where possible, what are the strengths of any positive impact that could occur, or the negative impacts which may need to be managed, or indeed the unexpected outcomes that could play a part in the change process.
As with Evaluation, an Impact Assessment can start with a Theory of Change (or Theory of Transformation) that outlines the key considerations for the implementation. This allows the assessor to compare the intentions and ambitions against the outcomes. In our model, evaluation is embedded in the process of change. It is a learning model that informs the progress of the change process. We are looking for other ways to ascertain the benefits or other or particular actions, beyond the transactional ROI approach. So, when we develop an impact assessment approach, we are also thinking of impact occurring at different points throughout the change process (short, medium and long term), and beyond the conventional measures.
A number of simple, familiar methods are being used to track and record impacts, such as canvases, mapping and pathways maps. In this MIRO board [https://miro.com/app/board/uXjVO5dOHUE=/?share_link_id=565130771848], four templates have been designed to work with. References to the original sources are on the board. identify
Make a copy of the board and save to your own MIRO space. Do not work on the master board.
1. MDF - EVALUATION + IMPACT CANVAS
2. MDF - RECORDABLE IMPACT OUTCOMES
3. MDF - PATHWAY TO IMPACT
4. MDF - IMPACT MAP
These templates can be used separately or together, depending on what you want to highlight in your implementation model. An example of how Energybank could be analysed is included on the board to show the working of each template. Depending on your case the stages will be different, as will the impacts on communities, systems, politics, society, and environments. Brief instructions are located below the templates.
MDF - EVALUATION + IMPACT CANVAS [above]
As with a business model canvas, this board compiles the composite parts in order to see the links and gaps that become apparent. By including all the people and situations that might become impacted in the project it is possible to cluster impacts to get a sense of the prominence and scale (who is impacted, how many people are impacted, and what is the significance of the impact, for example). We also see the internal and external factors at play. The design team, or the media attention are afforded a place within the mix.
Included here too, is an ‘Evaluation through line’ which gives a place for evaluative practices to be noted and their influence on and learnings from the project made visible.
Once analysed, all this information can help to assess the actual impacts, comparing them to the proposed impacts that were set out to guide the project. These can then be expanded into the other categories, such as social and environmental impacts. Depending on your case, these will be more or less prominent. As an example, it is likely that Homeful has more social impacts than environmental, where Plastics might be the inverse of that.
MDF RECORDABLE IMPACT OUTCOMES [above]
This board records the small, medium and prominent impacts across time: short, medium and long terms periods. A small impact that occurs in the short term frame might be a mention of the project in local media, while a medium sized impact in the mid or long term frame might be an influential research paper that leads to a new policy, which has continuing influence.
This board allows you to see that impact is not just the large and long term outcomes, but the smaller ones which lead to changes elsewhere. This draws on the new practices in evaluation, which could result in assessing the improved health of a community as a way to give value to the project outcomes. The impact is generated by many small instances, such as the co-design workshops, which accumulate to a well-functioning, coalesced community.
MDF - PATHWAY TO IMPACT [above]
The Pathway to Impact further develops the evolution of an impact, from the key research directions that help drive the project to the engagement activities to expected outcomes and arriving at significant impacts to note. In this example (above), the connections are linear. However, it is likely that they also intersect and/or cycle back to produce new insights.
This board asks you to view the process in different ways, in order to reveal outcomes that could be less visible otherwise. See the example of Pathway to Impact: Gender Equality in Advertising, by Dr Lauren Gurrieri on the MIRO board as an example of intersecting factors that produce significant impacts.…This board allows you to see that impact is not just the large and long term outcomes, but the smaller ones which lead to changes elsewhere. This draws on the new practices in evaluation, which could result in assessing the improved health of a community as a way to give value to the project outcomes. The impact is generated by many small instances, such as the co-design workshops, which accumulate to a well-functioning, coalesced community.
Another layer of interrogation is also shown on the boards (see red arrow lines). In this example, I have selected a minor impact posted in ‘Actual Impacts’ on board 1. Evaluation + Impact Canvas: Opening new frontier markets. An article from HBR is linked near the postit note that describes frontier markets as those that do not follow conventional market behaviours, but instead where a work-around might produce an interesting and successful innovation -in this case the Nollywood film centre (click on ‘source’ for the link to HBR)
A frontier market for Energybank might be opening up regional design as a viable industry located in the regions (see Gippslandia and Regional Design Service https://www.regionaldesignservice.com/ as examples). This is not an outcome that. was planned at the outset, yet it has a definite impact on both the regional economies as well as some design practitioners. A further conjecture is made on board 3, that this in turn leads to the development of Design in Regional Finance Peak Body, to consolidate this shift towards regionally-located practices.
The purpose here it to continue to look for impacts that might not be immediately apparent or valued.
In the third assessment task for Future Design Impact, you are required to produce an Evaluation and Impact report. You can use any or all of these boards, or the source boards, as a means to analyse and synthesise the evaluation methods and impact assessment you develop for your case. In addition to the boards, annotate the key insights to highlight your thinking.
Include an introductory piece on how you envision evaluation and impact becoming a critical aspect of the change process, with examples of how this might manifest itself through the process.
As stated earlier, evaluation using a HCD approach becomes a learning process as well as a lens on the work. In your report included a description of how evaluation is woven into the implementation model and how you then convert these findings into both ongoing learning models as well as the various impacts that result from the change project.
Dr Marius Foley
Program Manager, Master of
Design Futures
