Teaching Lab

Evaluating the Success of Professional Learning

HaMy Vu

Last week, as a part of our Learn with Teaching Lab series, we hosted a discussion focused on Evaluating the Success of Professional Learning. Specifically, we discussed how Teaching Lab evaluates the impact of professional learning in school systems, what the data says about how teaching and learning are changing in school systems we partner with, and how COVID-19 and school closures may shift our thinking around evaluating professional learning.

Why Evaluate Professional Learning? (Do We Need To Do It… Is It Hard?)

Every school system creates and provides professional learning of some kind to their educators. Sometimes it is done inhouse, sometimes a provider, like Teaching Lab, is hired. However, it is rare that the school system evaluates whether that professional learning is actually effective. Often, if evaluation is done, teachers are asked whether they thought it was useful or relevant. Those are good starting questions, and we use those questions, too, but that’s just the start.

We recognize that the word “evaluation” is scary to many. Evaluating professional learning may sometimes be an afterthought, even if you recognize that it’s important. And, many school and district leaders may feel they or their team lack the skill and expertise to become involved in rigorous evaluations. As a result, we’ve seen that school and district leaders either leave them to “evaluation experts” or neglect evaluation completely.

The good news is: good evaluations don’t have to be complicated. They simply require thoughtful planning, the ability to ask good questions, and a basic understanding of how to find valid answers. A strong evaluation plan can provide meaningful information that you can use to make thoughtful, responsible decisions about professional learning.

So, yes, if there is professional learning in a school system, it should be evaluated… and evaluation does not have to be hard.

Teaching Lab’s Theory of Change for Professional Learning

Evaluation does not start with the survey questions you want to ask; it starts with a solid theory of change. At Teaching Lab, our theory of change for our professional learning services is fairly simple:

If we…

Create best-in-class PL content
Build local capacity to lead and support that PL

And we…

Do it in a way that is aligned with our values and commitment to diversity, equity, inclusion, and anti-racism

Then…

Educator mindsets, skills, and practices will shift
System structures and norms will shift
Students will receive better and more equitable instruction

We start by getting clear about the very last step (the “then”), or the changes we would expect to see if we were successful in our efforts. Like backward planning as a teacher, ask first, what do you want students to learn at the end of a lesson or unit? Then, we plan out the content of the professional learning we’d have to create and/or deliver in order to be successful and achieve those changes.

Our Approach to Evaluation

Supporting the point that evaluation does not have to be hard, there is a great and widely used framework for evaluating professional learning or professional development, which was developed by Thomas Guskey. The Guskey framework has been so widely used because it is intuitive to educators and leaders. School systems themselves can take this framework and do their own evaluation. At Teaching Lab, we use the Guskey framework as a starting point and adapted it slightly to align with our theory of change. The framework describes 5 levels of evaluation: (1) participant reactions, (2) participant knowledge, (3) organizational change, (4) participant practices, and (5) finally student learning.

Essentially, the framework offers that if participants are satisfied with their professional learning experience, and if they gained knowledge from that experience, and if the school system supports them to put into their actions their learning, and if the participants actually change their practice because of what they learned, then student learning experiences will also change.

In our theory of change, we state our belief that good outcomes from our work mean educator mindsets, skills, and practices will improve. So in our evaluation, we use:

Surveys to track participant knowledge gained after attending our professional learning. We write quiz-like questions aligned to the content of our sessions, much like how a teacher would design an exit ticket at the end of a lesson.
Surveys to track educators’ mindsets about equitable instruction, to measure whether they hold high expectations for all of their learners.
Classroom instruction observations, to note whether the instructional practices are aligned to the instructional shifts expected by college and career readiness, and whether they’re changing.
Student surveys and analysis of student work from classroom tasks, to evaluate student learning experiences.

Teaching Lab’s Data Collection Efforts

In summary, when it comes to evaluating professional learning:

It is important to evaluate your professional learning.
Evaluation does not have to be complicated.
Always align your evaluation to your goals/theory of change.

Teaching Lab’s Impact: What We’ve Learned

By enacting our evaluation plan, Teaching Lab has learned a lot about how our 4,000 participants perceive our professional learning, as well as what impact our work has had on them and their 360,000 teachers across more than 30 school systems in 12 states.