Before You Collect Data, Ask Some Questions:
We take for granted now that schools and non-profits run on data. You’ll never hear a non-profit leader say “we have too much data”, or “we shouldn’t be collecting this or that”. However, before you say “let’s collect that”, ask yourself:
1) What’s the point? You might not hear it easily, but within each data request there is a subtle distinction between “Show me data so I can learn what works” and “Show me data so I can prove it works.” In one case, the purpose is learning and improvement, and the other it’s probably funding. Both purposes serve an organization’s mission. When bringing data in to a conversation, make sure you’ve got data that responds to the right question. For example, you may set a shoo-in goal for your funder that 75% of participants will achieve the outcome. When you meet with your internal leadership team or staff, let’s celebrate that outcome, but then look at the participants who did NOT achieve the outcome. In an improvement culture, we’re combing the data for problems: any participant who the program didn’t work for or who may need more interventions. In funder reports, we’re highlighting success and growth.
2) Is the juice worth the squeeze? I was chatting with a high school counselor recently who told me he spends three half-days a week inputting data, mostly case notes. I must have misheard. That’s a full day and a half each week! Why does he do this? Well, the boss needs it to keep the organization running. While I appreciate his commitment to the organization and data collection, that’s too much “squeeze” for something the funders aren’t actually asking for. When considering whether to collect data on something, know your plan for what you’re going to do with it. It’s not enough to say “It would be cool if we knew…”. We have to think about the time staff takes collecting data as a monetary cost to the organization. How much would you pay? You should pay for data that benefits participants because it will: (1) help fund your organization, (2) help staff do their jobs more effectively or efficiently (even considering that data collection), (3) play a key role in leadership’s decision-making process.
3) Are we going to look at the data? If there’s no plan and no team to review the data, and the purpose is learning and improvement (not funder-driven), don’t waste your time. Let everyone leave a bit early today. OR, consider how the data will inform key services before you collect it. If half of your organization’s work is helping young people plan for college, what do counselors need to know, and when? If the transcript report takes 5 minutes to run every time they meet with a student, the data can be used for reflection, but it’s not a tool. Do leaders and supervisors know how to talk about data, or value it? If not, get feedback to inform capacity building.
3 bullets is pretty standard. I’ll stop there. Anything I missed?