Thinking about evaluation before your project gets rolling

Most instructional technology initiatives in school are undertaken with a goal or set of goals in mind. Someone has received a grant to integrate a specific new device into their teaching, fundraising initiatives have resulted in new tools being made available to teachers in a grade or building, or the district has allocated funds for expanding its technology program through the creation of a new digital learning plan.

Somewhere in each of these scenarios is an assumption—articulated explicitly or not-- that by purchasing/implementing the new device(s) something desirable will result. Students will learn to collaborate through the use of online discussion and editing tools. Teachers will be able to share/store/upload lessons that will allow for the creation of more flexible, student-centered learning environments. Students will perform better on assessments after practicing their skills and receiving individualized feedback.

By the time a grant is approved, devices are installed and implementation has begun, however, goals are often the last thing on the minds of those responsible for getting the project off the ground. Thinking about outcomes frequently takes a backseat to a sense of urgency, excitement or the desire to “just get started and see how things go”.

At Sun Associates we work with clients to uncover, clarify, and in some cases create those important goals that underlie their projects before implementation begins. We encourage project teams to anticipate and plan for how they will explain the outcomes of their project work, and to reflect on the connections between the work that they do and the outcomes that they expect. Typically, we find that regardless of the scope of the initiative or project, pausing to think about evaluation up front will:

Clarify the connection between your project’s actions and its educational goals.

A project that has clarified its goals and actions will be able to say “We will provide/offer/do a, b, c, so that students/teachers are able to learn/develop/experience/achieve x, y, z.” Having such clarity up front will allow you to keep your project’s efforts focused and logically sequenced.

Create opportunities for team members to discuss and develop a shared vision for what a successful project implementation should look like.

A project that has taken time for this discussion will be able to say “We will know that our project is successful when we see or understand that students are doing x, y, z.” This sort of statement is the basis for creating indicators of project success. Indicators provide your project with the basis for assessing its progress during implementation as well as ultimate outcomes.

Consider how and what outcome data will be collected

The ability to identify outcome data such as “Students/teachers/ parents will report that the project has had impacts x, y, z ” or “Classroom observations will show students doing x, y, z” or “Student achievement data shows improvements in x.” will position you to respond to the inevitable requests for “data” that “show project impact”. Having this data will demonstrate your project’s commitment to providing concrete evidence of its success.

In short, spending some time framing goals, discussing indicators for success, and starting to consider what data you will collect from project activities allows your team to be well-positioned to evaluate your work. Planning ahead for evaluation in this way will also help you to describe your successes and identify areas for improvement as the initiative is scaled up or otherwise continued within your school or district. Even if you do not anticipate being challenged to ultimately report on project outcomes, spending time discussing and clarifying goals and outcomes at the outset of your project will help ensure that your efforts and project activities remain focused on supporting those outcomes.