How A/B testing works
When you have multiple active notifications of the same type in a property, TinyBell automatically splits traffic between them. Each visitor sees only one variant, and TinyBell tracks which one gets more clicks and conversions.
Setting up a test
- Create your first notification (Variant A).
- Clone it or create a second notification (Variant B).
- Change one element in Variant B (e.g. the headline, offer, or button text).
- Enable both notifications.
- TinyBell splits traffic 50/50 automatically.
Change only one element per test. If you change the headline AND the image AND the offer, you won’t know which change made the difference.
What to test
| Element | Example A | Example B |
|---|
| Headline | ”Book Direct and Save" | "Skip the OTA, Save 15%“ |
| Offer amount | 10% discount | 15% discount |
| CTA button | ”Book Now" | "See Rates” |
| Trigger | Page load | Exit intent |
| Image | Room photo | Pool photo |
| Urgency | No urgency | ”Only 3 rooms left” |
Reading results
Go to Statistics to compare performance:
- Impressions: How many times each variant was shown.
- Clicks: How many visitors clicked the CTA.
- CTR: Click-through rate (clicks / impressions).
Wait for at least 500 impressions per variant before drawing conclusions. Small sample sizes can be misleading.
Best practices
- Run one test at a time per property.
- Let tests run for at least 7 days to account for weekday/weekend differences.
- Once you have a winner, disable the losing variant and create a new test against the winner.
- Keep testing. There is always room to improve conversion rates.
