How A/B testing works
When you have multiple active notifications of the same type in a property, TinyBell automatically splits traffic between them. Each visitor sees only one variant, and TinyBell tracks which one gets more clicks and conversions.
Setting up a test
- Create your first notification (Variant A).
- Clone it or create a second notification (Variant B).
- Change one element in Variant B (e.g. the headline, offer, or button text).
- Enable both notifications.
- TinyBell splits traffic 50/50 automatically.
Change only one element per test. If you change the headline AND the image AND the offer, you won’t know which change made the difference.
What to test
Reading results
Go to Statistics to compare performance:
- Impressions: How many times each variant was shown.
- Clicks: How many visitors clicked the CTA.
- CTR: Click-through rate (clicks / impressions).
Wait for at least 500 impressions per variant before drawing conclusions. Small sample sizes can be misleading.
Best practices
- Run one test at a time per property.
- Let tests run for at least 7 days to account for weekday/weekend differences.
- Once you have a winner, disable the losing variant and create a new test against the winner.
- Keep testing. There is always room to improve conversion rates.
