Data quality is a scored line in research procurement now. Here are seven questions that make a vague answer obvious the moment you hear it. Everyone has language for data quality. Very few will commit to a number with a base attached. That leaves a buyer comparing claims that were written to be incomparable.
So aytm wrote a guide: Seven questions, in procurement language, each ending with aytm's own published answer. Which means the guide works on us too. Here's what the guide covers.
1. Where do your respondents actually come from?
Sourcing is the upstream variable, and everything downstream is compensating for it. A vendor whose sourcing isn't disclosed before fieldwork can blend in cheap supplemental sample mid-study, and by the time a cleanout rate surfaces the problem, the data is already in your decision. A strong answer names the proprietary panel and every supplemental partner, and tells you before the field opens rather than after. Ask what the screening actually turns up, too: aytm checks outside sample against PaidViewpoint's verified identity records pre-field, and in a single Q1 2026 month third-party duplicate attempts peaked at 46% of respondents in that outside sample.
2. How do you validate that a respondent is who they claim to be?
Every vendor has marketing language for fraud detection. Few will commit a removal rate against a disclosed denominator. The question also separates two things worth keeping separate: validation catches bad behavior from panelists already in, and verification proves a respondent is human. Both are required, and a vendor who treats them as one thing hasn't thought about it hard enough.
3. What quality monitoring runs during fieldwork?
Watching isn't catching, and catching isn't removing. A vendor who monitors without a described pathway from flag to removal has built a quality dashboard. Ask which behavioral dimensions get monitored, what threshold moves a respondent from flagged to removed, and where a human enters the loop.
4. What happens after the field, and against what base?
The most diagnostic question on the list. A cleanout rate measured against every entry that touched the survey looks flattering. Measured against qualified completes, it's a harder and more honest figure. Ask for the denominator, the sample size, the time period, and a per-study artifact rather than an aggregate rollup.
5. Can you reproduce the methodology behind any number you publish?
This one is about principle rather than current practice. A vendor who won't commit to publishing the formula, the denominator, and the period behind a claim is asking you to trust a marketing artifact. Ask whether they'd commit, and whether there's a cadence they hold to.
6. When AI is in your pipeline, who is accountable at each checkpoint?
The ICC/ESOMAR Code's 2025 revision requires named human accountability wherever AI sits in the pipeline. "Human in the loop" as a slogan doesn't satisfy that. Ask which AI tools are in the pipeline, who signs off at each checkpoint, and what the audit log looks like.
7. How do you enforce survey design quality before the panel sees it?
The most differentiated question, and the one most programs skip. Fatigue is a design problem, so a vendor who checks the data without checking the instrument is absorbing cleanout cost downstream and passing the quality gap to you. Ask whether survey-design scoring runs before launch, whether there are structural caps on length and complexity, and whether those caps gate a survey or merely advise it.
Why aytm answers all seven in public
Whoever writes the audit defines the conversation, so we'd rather the audit be symmetric. Every question in the guide ends with aytm's own answer, traceable to the figures we published in the Q1 2026 data quality benchmark report: post-survey cleanout at 5.4% of qualified completes, a 2.6% abandon rate against survey attempts, 1.6% incorrect demographics against verified respondents, and third-party duplicate attempts peaking at 46% of respondents in outside sample in a single Q1 month. Every one carries its base and its period, and any figure is open to a methodology audit on request.
The guide is the script. The report is the proof. Read either one and the other is the natural next step.
.webp)
.jpeg)
