TOOLS FOR USING DATA IN REAL-TIME: PART 1

by Kristen Gardner-Volle, MS, Evaluation Plus and
​Alexis Krause, MPH, Community and Cancer Science Network

Using data in real time can strengthen any initiative. The Community and Cancer Science Network (CCSN) is an initiative of the Medical College of Wisconsin and the American Cancer Society. CCSN brings together partners from the community and academic medicine to address the different outcomes for different groups of people in the breast and lung cancer space. Evaluation Plus (E+) is the developmental evaluation partner of CCSN. E+’s role is to support a learning using different methods to answer design questions and use data in real-time to execute on strategy.

One of CCSN’s initiatives is the Research and Community Scholars Curriculum (“Scholars Program”). This Scholars Program brings together community leaders and biomedical researchers to learn about cancer disparities and cancer research from different perspectives, discuss the role of science and lived experience with bias and racism, and experiment with community-academic partnerships for problem-solving.

The Case for Real-time Data Tracking
The first cohort of the Scholars Program started in September 2021. To recruit scholars, program leadership used a combination of general outreach and targeted recruitment strategies. Outreach included making connections with community leaders and researchers to create awareness about the program and to explain CCSN and its value proposition. More targeted recruitment strategies followed up on referrals as well as direct asks to potential applicants.
​
While we were hopeful we would meet our recruitment goals through these strategies, we still wanted to understand not just if the strategies were working, but how. To support the learning need, the team decided it needed a way to track the connections being developed, recruitment progress, and what challenges were surfacing. The solution: develop an easy-to-use and cost-effective tracking tool to capture the this data in real-time.

What is a tracking tool?
While we had made the case for real-time data tracking, we still need a way to gather the data. Tracking tools keep a record of what’s happening in a project. It can be as simple or as complex as required to meet the needs of the project. Some tracking tools are built into program management or other software packages. If you don’t have a software, you can easily build a tool using every day word processing or spreadsheet programs. The Scholars Program designed its tracking tool in Excel to document communication efforts and collect data to answer important evaluation learning questions, including:

• Are we on target to meet our recruitment goals? Why or why not?
• What questions do our networks have that could help us improve recruitment materials and strategies?
• Who is responsive to our outreach and who is not? And why?
• What kind of follow-up is needed in the recruitment phase?

How to Build the Tool
A quality tracking tool captures information consistently to make decisions about strategy. It can take time to identify and prioritize the most important data points to track. In the Scholars Program tracking tool, we wanted information like contact names, types of outreach efforts and outcomes of these efforts. To make it easy to use and consistent, we used drop-down options to get rid of inconsistencies in data entry. We knew our first version of the tool was not final, so, we met weekly during roll out to review the data being entered and make adjustments.

Getting the Most from the Tool
During recruitment, we used the tool to track the process and outcomes of different strategies. To make good decisions about strategy, its important to get good data and use it. One of our challenges was finding the time to get the data into the tool. To address this challenge, we appointed one person “information gatekeeper.” While the full team took responsibility for recruitment, the only person entering data was the information gatekeeper. The information gatekeeper regularly sent emails to team members with questions about their outreach. Additionally, team members copied the information gatekeeper on any recruitment emails.

To be sure we were using the tool and that the information was quality, E+ and the information gatekeeper met before each bi-weekly meeting to identify where more information was needed. We would then go into the program meetings with additional questions to address these information gaps and talk about what we were learning. These questions and discussions kept the project moving forward, but also allowed time to build out best practices for future recruitment.

Using a Tracking Tool in Your Own Work
Using this tool with the Scholars Program helped us manage processes and deadlines. It also helped us make near real-time adjustments to our plan. Further, it not only provided information about the success of our strategies, but helped manage multiple team members with different responsibilities for the project. By being intentional about the use of the tool, we also learned a lot about how to get the most out of our data. If you have a need for real time data, consider some of these ideas before building your tool.

Start with good data

• Define fields clearly. Don’t assume anything is common knowledge!
• Don’t leave cells blank. Instead, develop a way of noting missing data or data that is not applicable.
• Use drop down boxes rather than open text to maintain consistent data.
• Be clear and specific when asking for information from your contacts.
• Agree on a process for data entry. Once the process is decided, set up a training for people entering the data.

Be Flexible

• Focus only on most important information and how it will be used. Do not spend time and energy tracking data you won’t use.
• Know that it may take more than one communication to get the information you need.
• Offer multiple ways for people to provide data.
• Don’t be a perfectionist. It is okay to have missing data. The goal is to use the data you have right now. If you are using the data in real-time, it will get better.

Be intentional about data use

• Agree on the targets and assess your progress against those targets. The data collected mean little if you don’t know what you’re aiming for.
• Set a recurring time to review the data and engage your team in helping understand what it means. Each person should leave with clear action steps that will move the project forward.
• Realize that what you want to know may change overtime. Adjust your tool to serve your goals instead of your goals to serve your tool.

*Cross posted on Evaluation Plus*