It’s understandable as there is a reported lack of knowledge and experience needed to carry out such evaluation (Guskey, 2000, Goodall et al 2005). Furthermore a report by the Department for Education and Skills on ‘Evaluating the Impact of Continuous Professional Development’ (2005) found that only"CPD evaluation should not be seen as a tickbox exercise for governors." 24% of schools evaluate changes in pupil attitudes and a fewer than 10% of evaluation taking place rarely influenced the planning of any future CPD.
This worries me. It worries me in terms of ensuring that you get value for money from your training. It worries me in terms of planning using outcomes linked to CPD and, as a former school leader myself, it worries me in terms of ensuring the best education possible for learners. This is something I feel passionately about. With that in mind, what follows are a few suggestions that may help to structure your thinking when considering how to evaluate your CPD.
The purpose of evaluating the impact of CPD
One must consider from the outset not only the desired outcomes of CPD, but how these will be measured. What evidence will be obtained to determine and demonstrate that a positive difference is being made? Evaluation serves two purposes. Firstly, to identify whether the programme provides positive outcomes for a school (summative) and secondly, to identify how the programme itself can be further improved (formative). For me, building a long-term CPD programme is vital. It’s only when this is done that you can you accurately assess the gains, as well the next steps you can take.
Where to begin
Thomas Guskey's five levels of professional development offer a template when thinking about CPD evaluation. These are as follows:
Level 1: Participants reaction: Will the information be useful? Did the material make sense? Was the leader knowledgeable and helpful?
Level 2: Participants learning: Did the learner obtain new knowledge and skills?
Level 3: Organisation, support and change: What was the impact on the organisation? What support was provided to initiate change/s.
Level 4: Participants use of the new knowledge and skills: How does the participant apply new knowledge and skills. How is this assessed?
Level 5: Student outcomes. What is the impact on learners? Achievement, Confidence, Attendance, Behaviour, Self-esteem.
Ask yourself whether the type of data you already use can fully answer the questions linked to each of the levels. If not, what changes can be made to make sure that they do? I’ve found this to be a useful activity to focus attention on a school’s current CPD programme and how evaluation can go towards improvement. For instance, research shows that the majority of CPD evaluations take place at stage 1 (participant reactions), straight or soon after the CPD programme has taken place. Although obtaining participants reactions is very important, only relying on stage 1 means that the evaluation is often brief, subjective and difficult to interpret. It is important to consider when planning your professional development how each stage of evaluation can be put into practice, acted upon and the evidenced. Each level should build on what has come before.
Changing what evaluation means
Evaluation can often be a frightening prospect, however it should never be avoided due to fear of obtaining evidence that might show undesirable outcomes. Evaluations that focus on how teachers’ practice and embed new knowledge and learning from a professional development programme are invaluable for determining impact as well as the time and money spent. Furthermore, CPD evaluation should not be seen as a tickbox exercise for governors, inspectors or other external stakeholders. Investing in this process is about really wanting to improve pupil learning and the quality of teaching in your school.
- When planning CPD, determine from the outset how you intend to evaluate its impact.
- Focus on measuring the difference it can make to teacher practice and student outcomes, rather than just the CPD activity itself.
- When using Guskey’s Five levels framework for evaluating professional development try starting with level 5 first and working backwards.
- Use a range of quantitative and qualitative data– Questionnaires, interviews, focus group meetings, observations, feedback sheets, reflection logs etc. Consider carefully the nature of questions, rigorous baseline.
- Involve all participants in the process of the evaluations from the start. CPD evaluations are not the only job of the senior team.
- Do not just add evaluations practices at the end of your CPD programme or as an add-on. Determine from the start what children will learn differently as a result of the CPD activity.
- Do not just evaluate participants perceptions – this could lead to bias and very subjective results.
- Avoid focusing just on the professional development programme, the material and the training itself.
- Make CPD evaluations burdensome. With the right training, a practical and collaborative approach with the use of rigorous tools this can become quite straightforward.
- Do not begin any form of evaluation until you are clear on:
- The level of questions you will address (at each of Guskey’s levels),
- How the information will be gathered,
- What is measured,
- How this information then be will be used.
Despite its challenging nature, the long-term commitment and critical planning needed, evaluating the impact of CPD is critical if maximum gains for students are to be achieved. Measuring the impact of training is an integral step on the journey towards ensuring the best training and the best outcomes.
Craft, A. (2000) Continuing Professional Development: A practical guide for teachers and schools, London: Routledge Falmer.
Edmonds, S. and Lee, B. (2001) Teacher Feelings About Continuing Professional Development, Education Journal, 61, 28–29.
Goodall, J et al (2005) Department for Education and Skills, Evaluating the Impact of Continuing Professional Development (CPD)
Guskey, T.R. (2000) Evaluating Professional Development, Thousand Oaks, Ca.: Corwin Press.
Hargreaves, A. (1994) Changing Teachers: Changing Times, Toronto: OISE Press.
Harris, A et al (2006)- What Difference Does It Make? Evaluating The
Impact of Continuing Professional Development In Schools, Scottish Educational Review, University of Glasgow, Volume 37.
Ofsted, 2006 ‘The Logical Chain’
Porritt, V (2005) London’s Learning, developing the leadership of CPD, department of Education and Skills.
How do you handle CPD in your school? Let us know below.