At Sun Associates, we pride ourselves on finding ways to help educators make the most of freely-available resources. A large part of our online presence is devoted to providing links to the various tools and resources we have developed over time and now offer freely to educators. When these tools are things such as surveys, it is only natural for the online community to ask “Can I use your survey in my evaluation or audit project?”.
The short answer to this question is “Yes!”. But as is so often the case in life, there’s something of a caveat attached to this answer. This caveat has nothing to do with permissions or copyright (all of our tools are licensed under a Creative Commons Attribution ShareAlike license), but rather is connected to how one intends to use a tool such as a survey. More specifically, we ask you to take a moment to consider if the chosen tool is really appropriate for the intended use. We find that thinking a bit about how an instrument such as a survey fits into an overall plan for evaluation and assessment will ultimately:
- Save you time
- Be more respectful of those from whom you are collecting data
- Generate more meaningful evaluation findings
Let’s take a moment to consider each of these points.
Saving Your Time – No one ever has enough time to accomplish all that they wish to do. This is certainly the case with busy Technology Coordinators and School IT staff (our typical clients) who are most often dealing with evaluation and assessment as an add-on, “Oh, could you do this too?”, last minute task. In such a situation, probably the last thing that you’d want to do is to wind up using a survey (or focus group protocol, observation template, etc.) that generates data that is not directly connected to what you actually want to know about your program or initiative. It makes little sense to collect data that you do not ultimately need, but this is exactly what can happen if you use a generic instrument that is not connected to your own program’s goals and objectives. Remember that data collection is just part of the task of using data to assess and drive decision-making. Any data you collect must also be analyzed and reported, and this of course is a time-consuming activity. That’s your time that’s being consumed. Wouldn’t it be better to only collect that data which you will actually need?
Respectful Data Collection – Just as you do not want to waste your time collecting and analyzing data that you do not need, you also do not want to waste the time of your teachers/students/parents/community in providing information that is not ultimately relevant to your program’s evaluation needs. “Survey fatigue” is very real in many schools and organizations, and this does not just come from too many surveys, but also from surveys that ask questions that seem irrelevant. Your population’s desire to be helpful has limits, and if your survey seems like a data “fishing expedition” to respondents, survey fatigue can set in quickly.
Generating More Meaningful Findings – As someone aiming to use data to evaluate or assess a program, clearly your bottom line is to generate meaningful findings. Data does not stand alone as a “finding”. Rather, data needs to be analyzed and assembled to answer questions about your program’s performance. That analysis is what generates findings…and findings are the ultimate product of any evaluation effort. We often think of data as filling in a picture of program performance much like one would complete a paint by numbers picture. Obviously, you need to have all of the colors (data) that match the numbers on your picture template (indicators, in the parlance of evaluation). Therefore the logic behind selecting and creating data collection tools is to ensure that your tools provide all of the data that fills in your picture. If you collect data that you do not need, and miss collecting data that you do need, then your findings will suffer. Yes, this is common sense, but it is easy to lose track of this logic in the rush to gather tools and to start collecting data.
Where does this leave the erstwhile evaluator/data collector who is simply looking to “use that survey”? Well, as we have said, you’re free to use our materials…and no doubt you have found other free surveys out there on the ‘web that also look like they would work for you. But, before you just use these resources wholesale, take a little time to think about 1) what you really want and need to know, and 2) how you might edit various tools to meet your needs. Considering these issues is in fact part of a process for creating and conducting effective program evaluations and audits. While the notion of a “process” can be scary and may seem overly time-consuming, we work with educators day in and day out to engage effectively and efficiently in evaluation work. In fact, we have webinars and various resources about how you can become an effective evaluator of technology programs and initiatives. So if you want to want to use our surveys, feel free! And also take a look at some of our other resources that can help you make the best use possible out of those free resources.