We’ve been reworking, revising, refining, and reviewing our evaluation plan for Studio to School for the current year all fall! And we’re finally ready to share it with you, along with some reflections on what we’ve learned about the evaluation itself.
We invite you to take a closer look at the full evaluation plan and to let us know if you have questions. This is a working document – we expect that things may change as the year progresses. It is also an internal document – a chance to see “behind the curtain” if you will. As such we ask that you don’t share this beyond the learning community without asking first so that we can provide sufficient context.
Here’s a little reflection and some highlights from the plan:
As you know, we’re taking a developmental approach with the Studio to School evaluation. This means that we’re adapting as we learn, just as the projects and broader Initiative are.
This approach has not been without challenges for all involved! We have struggled at times (along with some of you) with the “blurry” aspects of the work. Wouldn’t it have been lovely to be able to tell everyone exactly what to measure and when from the beginning of year 1? Maybe! However, we can also see that we would have missed opportunities to adapt as things progressed had we planned everything out in detail from the beginning. The project evaluation planning work, for example, is something we didn’t foresee at the beginning of the Initiative, but that by August of the first year had become important to include in our work. It’s value has only grown since that time.
Here are a few things we’re doing differently this year, and why:
- No more Topical Action Research groups! While this evaluation and learning community activity had great promise, most groups were unable to squeeze this into their already busy work of the Studio to School projects and other important learning community activities. We are keeping the work that some groups were able to accomplish on the website here, and will be working to build on it as we conduct literature review and other evaluation activities this year.
- Shorter evaluation visits! The long, detailed conversations we had with each project team last year were very valuable in helping us learn how the projects were progressing, what challenges the teams were facing, and in getting to know the teams and projects better more generally. We also really loved getting to see many of your arts education activities in action. However, this year, given that Michelle is also making a round of visits, and that teams are visiting one another, we’re shortening our visits. You’ll hear more from us soon, but can expect that we’ll
- No collaboration survey! We planned early on to not conduct this survey every year, but instead every other year. We’ll talk more about your efforts to collaboration within your teams as well as engage the community again when we visit, to update what you’ve shared in your e-journal post following the rendezvous in Sisters. And you can expect another survey next year!
And here are a few things that we’ve built into the plan for this year, based on what we learned last year:
- We deepened the project evaluation planning began in year 1 and are moving projects toward more detailed planning and evaluation work. We’re working now to determine what trends we’re seeing, especially regarding shared needs for support, and we’ll spend time during our visits reflecting and working on those evaluation plans. It seemed a natural next step to add in the more detailed rubric this year – each of the items we asked teams to include in that rubric are important both to OCF and to the project teams given what everyone learned in year 1 and everyone’s goals for Studio to School.
- We’re engaging in literature review around the topics that you have called out as most important, including quality in arts education, arts learning assessment, and social-emotional learning. We plan to share what we’re learning at the March rendezvous and will be interested in your reactions and feedback. We hope to combine what we learn from the literature review with what we’re learning through the Studio to School projects to develop “learning briefs” that outline what existing research says (and doesn’t) and how that is experienced by the project teams (or not)!
- E-journaling continues, and the website is significantly improved this year. Your e-journal posts will continue to be a really important source of information for our evaluation work; we’ve built out the topics for this year in response to what we learned was most important last year (e.g. arts learning, which you’ll journal about in spring 2016). These are also a wonderful way to stay abreast of what’s happening in your projects – we really appreciate this glimpse into what you’re doing and learning. We also heard you loud and clear that the initial website set up was overly-complicated and clunky to use at best. We’ve redesigned things to make everything easier to find and are now making sure (finally) that all learning community team members have access to the website. We hope that everyone will be able to take greater advantage of what is shared this year.
Now, we’re curious about your take on our evaluation (and evaluation planning) thus far. Do you see other adaptations that we’ve made between year 1 and year 2? Any surprises about the evaluation process so far? Anything you think we’re not paying attention to but should be?