Welcome to this guide on how to set up a campaign A/B test in charles. A/B testing is a great way to understand what works best for your campaigns and it's straightforward to get started. Follow these steps to create your first A/B test campaign.
Step 1: Create a Base Campaign
1.1 Enter Campaign Details:
Start by entering a campaign name, add "Variant A" to the name for easier identification. You can also add a coupon code if relevant, the code will be used to attribute revenue to the campaign(s). Choose the template type that suits your campaign needs. If you want to make a general announcement, a regular template works best. In case you want to announce a new launch of new products, we recommend to use a carousel template.
1.2 Create the Template
Design your campaign template. This is where you decide on the content, such as text, images, and layout. When you are happy with the message, submit it to WhatsApp, it should be approved in no time.
1.3 Configure the Audience
Select the target audience for your campaign. This step is crucial as it determines who will receive your message. Use different property, opt-in or event filters to create the ideal audience. In case you already have a segment you want to use for this campaign, use the radio button on the top to select the segment.
1.4 Save for later
Save your campaign by clicking the button "Save for later" in the top right corner.
Step 2: Duplicate and modify for A/B testing
2.1 Duplicate the Base Campaign
Go to the campaign overview and duplicate your base campaign. This creates a copy that you can modify for testing purposes. Add "Variant B" to the name for easier identification.
2.2 Modify for Testing
To learn about A/B tests and see some best practices, go to the section below.
✏️ If you want to test different content (e.g., image vs. no-image), go to step 2 (Edit template) and adjust the template. For example, change the image, remove the image, change the wording of the message or find a different button label. These are all good ways to optimize the performance of your campaigns. Next, schedule both campaigns for sometime in the future.
👨👩👦 To test a different audience, go to step 3 (Target audience) and change the audience configuration. Next, schedule both campaigns for sometime in the future. If you want the same audience to be split into 2 equally sized lists, please create your list or segment first, and then use the "Split" feature to create the lists. You can then pick one list per campaign variant.
⏱️ To test different send-out times, go to the final step of each campaign and schedule the campaign for another time or day.
Step 3: Analyze A/B Test Results
Identify Campaigns: After the send-out, go to the campaign overview to find the two campaigns from your A/B test. You can use the date filter in the top right corner of the table.
Look at key metrics such as click-through rate, conversion rate, and revenue per recipient. These will help you determine which version of your campaign performed better.
Best Practices for A/B Testing
Test One Variable at a Time: To understand what causes changes in performance, only alter one element of your campaign at a time (content, audience criteria or time of send-out).
Use a Large Sample Size: Ensure your audience size is sufficient to detect meaningful differences.
Run Tests for a Sufficient Duration: Allow enough time for your audience to interact with the campaign, providing reliable data.
Focus on Key Metrics: Concentrate on metrics that align with your business goals, such as conversion rates or revenue per recipient.
By following these steps and best practices, you'll be well on your way to mastering A/B testing with charles. Happy testing!