Data Analysis
Data AnalysisTestingDecision MakingA/B Test Results Interpreter
Reads raw A/B test numbers and returns a statistical significance verdict with a clear ship or kill call.
Best For
- •Solo marketers running their own landing page tests who don't have a statistician to tell them if results are real.
- •Growth-focused founders who want a fast significance check before committing engineering time to ship a variant.
- •Freelance CRO consultants who need a repeatable framework to present test results to clients clearly.
Prompt Template
You are a data analyst interpreting A/B test results. I'm going to paste the raw results. Tell me whether the results are statistically significant and what to do next.
Output:
1. Test Summary: what was tested, control vs. variant
2. Results Table: side-by-side metrics for control and variant
3. Statistical Significance: is the difference statistically significant at 95% confidence? Include confidence level, p-value estimate, sample size assessment
4. Practical Significance: even if statistically significant, is the effect size large enough to matter?
5. Verdict: SHIP IT / KILL IT / KEEP RUNNING / ITERATE
6. Caveats: anything that could invalidate these results
7. Next Test Suggestion: what to test next based on what you learned
Test results:
[PASTE RESULTS, include: sample sizes, conversion rates, confidence intervals if available, test duration]
Pro Tip
Always include the test duration and total sample size, not just the conversion rates, since a result can look dramatic but still be statistically meaningless with too few visitors or too short a run.
Example Output
A sample of what this prompt produces once you fill in the placeholders.
An analysis of a homepage CTA test finding a 33% relative CTR lift with 97% confidence, a SHIP IT verdict, and a caveat recommending one more week of data to rule out weekly traffic cycles before fully committing.
How to use this prompt
- Copy the prompt template using the button above.
- Paste it into your preferred AI assistant (ChatGPT, Claude, Gemini, etc.).
- Replace all bracketed placeholders like
[TOPIC]with your specific details. - Send the prompt and refine the output as needed.