AYTM Sample Engine: a practical guide

Last updated: 2025-11-14

TL;DR: The AYTM Sample Engine lets you define who you want to talk to, control incidence and quotas, and get reliable completes without wrestling with spreadsheets. Start by defining a clean target audience, estimate incidence, then use quotas and quality controls to make sure the people in your study match your brief.

What is the AYTM Sample Engine?

The Sample Engine is the part of AYTM that turns your audience definition into actual respondents. It manages:

• Targeting (who can see your survey)
• Incidence (how many people qualify)
• Quotas (how many completes you want in each group)
• Delivery (how quickly completes are collected)

Once configured, your survey can launch to exactly the audience you need, at the sample size and balance you specify.

Step 1: Define your core audience

Start by answering a simple question: “Who must be in this study for the results to be useful?”

Common dimensions include:

• Location (country, region, city, or other geo)
• Age ranges
• Gender or other identity variables
• Basic demographics (income, household, education)
• Category behaviors (buyers, users, decision makers)

Most of this can be set through Sample Engine targeting and a short screener in your survey.

Step 2: Decide on sample size

Your sample size depends on the type of decision you’re making and how granular you need to cut the data. As a simple rule of thumb:

• n=200–400: quick reads and light segmentation
• n=400–800: more reliable segment comparisons
• n=800+: deeper cuts and multiple subgroups

If you need to compare multiple concepts, ads, or segments, plan your sample so each comparison cell has enough respondents to be interpretable.

Step 3: Estimate incidence and qualification

Incidence is the percentage of people who qualify for your survey after screening. It affects cost, timing, and feasibility. For example:

• If 50% of people qualify, incidence is 50%.
• If only 1 in 10 qualify, incidence is 10%.

On AYTM, you can often use past studies or category benchmarks to estimate incidence. If you’re unsure, build a lean screener and discuss expectations with your internal team or account contact before launching.

Step 4: Set quotas where they matter

Quotas ensure you get the right mix of respondents. You might need quotas for:

• Age or life stage (e.g., even split across key ranges)
• Gender or identity
• Users vs. non-users of a brand or product
• Regions or markets

Good quota design focuses on the few dimensions you truly need balanced. Avoid over-quoting on small, overlapping groups, which can slow fieldwork and increase cost.

Step 5: Align sample with your screener

Targeting in the Sample Engine and screening in your survey should work together:

• Use targeting for broad characteristics like country and age.
• Use screener questions for more detailed qualifications (e.g., category behaviors, decision roles).
• Keep the screener short and clear—complex logic can lower incidence and frustrate respondents.

After launch, monitor early completes to confirm that screener logic behaves as expected.

Step 6: Monitor fieldwork and quality

Once your survey is live, keep an eye on:

• Fill rate: are completes coming in at the expected pace?
• Quota progress: are some cells filling too fast or too slow?
• Data quality: are there signs of speeding, straightlining, or poor open-end responses?

AYTM’s quality controls and monitoring tools help identify low-quality responses and keep your final dataset clean.

Tips for getting the most from the Sample Engine

• Start simple: define the smallest set of must-have criteria before adding “nice-to-haves.”
• Be realistic about rarity: very narrow or rare segments may require longer field times or alternative approaches.
• Document your plan: write down your target, incidence assumptions, and quotas so future waves can replicate your setup.
• Use past projects as templates: clone successful setups for similar audiences to save time.

FAQs

What if my incidence is much lower than expected?

First, double-check your screener logic. If it’s correct, consider broadening criteria slightly or adjusting timelines and budgets to reflect the true incidence.

Do I always need quotas?

No. If you’re targeting a simple audience and don’t need specific splits, you may not need quotas at all. Use them when representation or balance across groups is important for your decisions.

Can I change sample settings mid-field?

Some settings, like quotas or daily caps, can be adjusted while a study is in field. Structural changes to who qualifies may require pausing and carefully updating the survey and targeting.