In this Engagement Matters post, Suzanne Spicer shares her top tips for evaluating the success of your public engagement activities.
Evaluation can be perceived as a daunting task but when used correctly, it is an effective tool to reflect on and improve your public engagement activities, and to determine the value and worth of evidenced impact.
How to start: Whenever possible you should plan your evaluation when you are planning your public engagement activity. Begin by considering why you are undertaking the evaluation. It is important to be clear of your purpose right from the start.
Write an evaluation plan: To keep focused have an evaluation plan, a step-by-step guide which summarises what you are going to do and how you will undertake the process.
Top tip: keep things simple.
Include the following in your evaluation plan:
1. Aims and objectives
Your aims are what you want to achieve overall and your objectives are how you will achieve your aims. It is important to keep them SMART so ask yourself:
Specific: do they state what will you do and with whom?
Measurable: can you measure their success?
Achievable: do you have enough time and resources to achieve them?
Relevant: do they meet your aims?
Time-bound: do they include timescales?
Top tip: only have one or two aims and up to five objectives.
2. Audience and stakeholders
It is important to identify who will be involved in your evaluation and what challenges may arise when engaging with them.
Top tip: if appropriate, involve the public in your planning.
3. Evaluation questions
These are the questions you want answers to. They should not only measure outputs (the results of your activity) but also outcomes (eg. an increase in awareness, the development of skills or a change in behaviour/practice).
Top tip: have between 2-6 questions and ensure they relate to the evidence you can collect.
4. Collecting evidence
Think creatively about how you will gather your data and how you might build it into your activity, so that it is easy for you to collect, and your public want to engage. There are a variety of methods you can use from the more traditional such as questionnaires to more creative methods such as graffiti walls or using a voting app. You do not have to evaluate everyone and everything – you can sample. Also remember to capture perspectives from everyone involved (including yourself). And if you are measuring impact then it is important to create a baseline from which you can evidence any change.
Top tip: if you are engaging with a new audience, look for advice from other sectors.
Whichever method you use, you will have to ask questions. Use a mixture of open (ask for open-ended responses) or closed questions (give a series of options for the participant to select) and ensure your questions are clear and easy to understand. Check that none are leading or biased and try to avoid asking people to predict their behaviour too far into the future. Also avoid asking multiple questions in one.
Top tip: test your questions out beforehand.
5. Analysing your evaluation data
Decide how much data you want to collect and how much time and capacity you will have available to analyse it. With quantitative data (numbers) you can use a spreadsheet to analyse your data. With qualitative data (words and images) you can group similar responses into categories that can then be counted and reported or expressed as a percentage.
Top tip: you can use audience quotes to illustrate points but check they are representative and give a balanced perspective.
6. Using your findings
Once you have analysed your data then you should interpret what you have found. List your key findings both positive and negative and link them to your evaluation questions and critically reflect on what you have learned.
7. Sharing your learning
Finally, think about how you will use and share your findings. Identify your key audience and who else might be interested and which formats you might use, such as a report or an infographic. Whatever platform you use, be clear about your key messages and share what you have learnt, both the positive and negative.
Some useful resources
You can find some useful evaluation links and resources here.
See also: Suzanne Spicer, The nuts and bolts of evaluating science communication activities, Seminars in Cell & Developmental Biology, Volume 70, October 2017, pp. 17-25 available from: http://www.sciencedirect.com/science/article/pii/S1084952117304494
Posted on behalf of Suzanne Spicer FRSA, Social Responsibility Manager, Office for Social Responsibility, The University of Manchester
Twitter: @UoMEngage | @csusies | #EngageMatters