Asking the Right Data Question

October 22, 2025

Comments

Most campus meetings start the same way: someone asks, “How are we doing on retention?” or “Is the funnel healthy?” Dashboards open, people scroll, and the hour ends without an answer. A business prompt is not yet an answerable data question. Part 2 shows how to close that gap. You’ll learn to frame a question in one statement. You’ll choose the right population, granularity, and timeframe. You’ll show your math (numerator, denominator) and name a baseline. Then you’ll run a few quick trust checks, so the data you present is reliable. With new cycles starting and reporting deadlines stacking up, getting to a clear question and a quick trust check can save a week of back-and-forth. 

Our approach is specific to Evisions, and it’s grounded in frameworks you may already know. Jisc’s work on building digital capability highlights role-appropriate skills and confident practice. The European Commission’s DigComp 2.2 offers practical examples for working safely and critically with data and new technologies. For a synthesis across sectors, Ridsdale and colleagues summarize data literacy as the ability to collect, manage, evaluate, and apply data, and this arc maps directly to the steps we use here. 

If Part 1 gave you a shared language and a simple campus data map, Part 2 gives you a repeatable way to turn questions into decisions. Here’s what this post covers and how to use it in your next meeting. 

What you will learn 

Over the course of this post, we’ll cover how to: 

  • Frame a decision question in plain language and give it a deadline. 
  • Choose the right population, granularity, and timeframe, so the analysis you complete answers the question. 
  • Show your math by writing the numerator and denominator for any rate and naming a baseline or target. 
  • Trust the data by running a lightweight trust check that covers collection realities, cleaning and validation, and a one-line lineage from source to output. 
The Question-to-Query Path 

A one-page scaffold that turns a vague prompt into an answerable, reproducible question that leads to an answer. It forces agreement on some key areas where you could have later disagreements: the data question, population, granularity, timeframe, the exact metric, baseline, and source. It also includes a couple of trust items…all so you avoid circular debates and mismatched numbers. 

  1. Define the Ask 
    • Initial prompt – copy the original question in their words 
    • Problem context – one sentence about what prompted this work. 
    • Primary stakeholders/audience – who the final output is for. 
    • Deadline – when the stakeholders need the data. 
  2. Shape the Question 
    • Population – the inclusions and exclusions that define who is counted. 
    • Granularity – the unit per row (student per term, applicant per term, student-course section). 
    • Timeframe – the window you will use (Weeks 1–4, Fall 2025 census). 
    • Metric definition – the numerator and denominator in plain language. 
    • Baseline or target – the comparison you’ll use (last term, three-year average, goal). 
    • Slices to compare – dimensions you will actually discuss (program, modality, Pell, residency). 
  3. Write the Question 
    • Data question – the specific question answerable by the data available.  
  4. Data & Trust Details (for reproducibility) 
    • Source of truth and freshness – the system or view you’ll use and the refresh window. 
    • Required fields and exclusions – IDs, dates, status codes, and any test or training rows to ignore. 
    • Data collection realities – practical caveats that move numbers (late postings, batch timing, ID merges). 
    • Privacy and small-n – the suppression threshold (e.g., n < 5) and any masking rules. 
    • Mini-lineage – one line listing Source → Transform(s) → Output with table or view names. 
  5. Execution & Output 
    • Analysis plan – the one or two steps you’ll take for the analysis. 
    • Output and owner  the artifact you’ll produce and the person who owns it. 

I know it may seem like a lot, and you don’t have to use it all for your first try. I recommend running through it live in a meeting. Answer the parts that come easily, circle back to the ones that don’t, and timebox yourself to 15 minutes to see how far you can get. The more you use the question-to-query path, the easier it will get. Be sure to attach the results to your data request ticket or task! 

Trust the Data: Five Quick Trust Checks 

Five super-quick (5 minutes or less) checks you run during the data artifact creation process or before you publish a number or a chart. They make sure the result matches what you wrote on the question-to-query path, and that someone else could reproduce it. They also prevent the most common errors and align with the collect, manage, evaluate, apply progression you’ll see in the research. Think of them as “did we count the right thing, in the right window, the right way?”

  1. Keys & counts match the grain 
    • Do: Confirm the unit per row is unique (e.g., one row per student-term; joins didn’t duplicate rows). 
    • Pass looks like: Distinct count at your grain equals the row count; totals behave when you group. 
  2. Window match what you wrote 
    • Do: Verify the filter window (Weeks 1–4, Fall 2025 census, rolling 7 days) match your Path. 
    • Pass looks like: The date filter in your query/tool matches the written window.
  3. Duplicates & test rows removed (population rules applied once) 
    • Do: Drop test/training records, stale IDs, and accidental duplicates; re-check inclusions/exclusions. 
    • Pass looks like: No “TEST” names, no duplicate IDs at your grain, population lines up with the spec. 
  4. Tie out to the source of truth (within tolerance) 
    • Do: Compare your total to the official view (SIS census view, authoritative warehouse table). Name your tolerance. 
    • Pass looks like: Within agreed tolerance (e.g., ±0.5% or ±1 record) given the refresh window you stated. 
  5. Privacy: small-n and sensitive mixes 
    • Do: Suppress or aggregate where n < 5; avoid combinations that could re-identify individuals. 
    • Pass looks like: Tiny cells are hidden/rolled up; you note the rule in the footer (e.g., “Small-n suppression: n<5”). 

Those are just five quick data trust checks you can perform, but we’ve compiled a more complete list in our Data Trust Checklist. Download it today and make it your own for your institution.  

Common Pitfalls to Avoid 
  • Moving denominators – don’t change who is in the denominator between slices or weeks. 
  • Window mismatch – the query window must match the stated window. 
  • Silent joins – duplicate rows through joins are the number one cause of mismatched counts. 
  • Hidden freshness – put the refresh cadence and last refresh timestamp in the footer of any reports or dashboards. 
  • Small-n risk – apply suppression and say so in writing. Know the reason if people ask. 
  • No owner – name the person who will produce the artifact and the person who will make the decision. 
How To Get Started 

Download our Question-to-Query Path and Data Trust Check. Add them as a discussion point for your next data team meeting.  

Consider starting by: 

  1. Give the Question-to-Query Path a try in at least three different meetings.  
  2. Complete the Data Trust Check before releasing your next two reports or dashboards. 
  3. Find at least one collaborator who can help you gain traction at your institution. 

You don’t have to do everything at once. Start small and remember that perfect is the enemy of good. Give some of your ideas a try and refine as you go. Building small habits now can offer big returns as the academic year continues.   

If this still feels overwhelming or you’d like a little extra support, don’t hesitate to reach out to your Strategic Solutions Manager. Our Client Experience Team (CET) is always here to partner with you and help bring your reporting and data analytics goals to life. 

Where the Field Guide Goes Next 

This is the second post in our six-part series on Data Literacy. Turning a question someone asks into an answerable data inquiry helps turn your campus into a more data-empowered institution. It’s just one step on your data modernization journey! While each installment builds on the last, you can use Part 2 on its own and see immediate results. 

Join us in two weeks for Part 3: Metrics That Matter! 

Allen Taylor
Allen Taylor
Senior Solutions Ambassador at Evisions |  + posts

Allen Taylor is a self-proclaimed higher education and data science nerd. He currently serves as a Senior Solutions Ambassador at Evisions and is based out of Pennsylvania. With over 20 years of higher education experience at numerous public, private, small, and large institutions, Allen has successfully lead institution-wide initiatives in areas such as student success, enrollment management, advising, and technology and has presented at national and regional conferences on his experiences. He holds a Bachelor of Science degree in Anthropology from Western Carolina University, a Master of Science degree in College Student Personnel from The University of Tennessee, and is currently pursuing a PhD in Teaching, Learning, and Technology from Lehigh University. When he’s trying to avoid working on his dissertation, you can find him exploring the outdoors, traveling at home and abroad, or in the kitchen trying to coax an even better loaf of bread from the oven.

Related Posts

Pie Chart – Headcount by Age

The Chart  This is a pie chart, useful when you want to show how a small set of categories make up a single whole. Each slice represents a non-overlapping group (age band), so you can see the composition quickly. For readability, keep the number of...

Foundations of a Campus Data Ecosystem

For more than 25 years, we’ve focused on one thing: helping colleges and universities use data and technology to move real work forward. Today we’re excited to introduce the Field Guide Series, a six-part collection of short, practical modules you...

0 Comments

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *