A/B Testing

Test two videos against each other to optimize your video marketing efforts!

A lot of variables go into a successful video. But how to know which factors play into that success? With Wistia’s A/B testing feature, available on the Advanced plan, you can set two videos up to be tested against each other and see how they fare with your audience! Viewers will see one version or the other, randomly selected, and we’ll show you the two videos' performance side-by-side. When your test is over and you’ve picked a winner, we’ll automatically show that winner to all viewers. A/B Testing is accessible to the Account Owner and all Managers on the account.

Setting up a Test

To set up an A/B test, you’ll need two videos to test against each other.

Tip
Most likely, you’ll want to copy a video so that you can make small tweaks, to compare against its original version - our Move & Copy feature can help with that! Move & Copy is available on all plans.
Then, head to the Customize panel of one of your videos, and make the changes you want to test on. It’s easiest to be certain of your results if you’re just testing on one factor (for instance, a video thumbnail versus a still image thumbnail, or using a Call to Action versus an Annotation), but you can make the two videos as different from each other as you want. Once you’ve got your two videos set up the way you want them to be, it’s time to create the test.

From the Stats menu, choose A/B Tests. abtesting-menuYou can also find the option to select A/B Tests in the ▸ Video Actions menu.

Click Create Test to start a new test, and be sure to edit its title so that it’s clear as to what you’re testing. You can change the title by hovering over the current name and clicking the Edit button that appears.

create-ab-test

Click Choose Video on the left to select your first video - the “A” in “A/B testing” - then repeat on the right to select the “B” video. Then, click Start A/B Test to get the specialized embed code you’ll need. Click the View Embed Code button to see and copy that embed code. Paste it into your website editor, and watch the stats roll in!

Checking Your Stats

Naturally, you’ll want to see how your two videos stack up against one another. It’s easy to check that in the A/B Testing tool. When you choose A/B Testing from the Stats menu, you’ll see all of your current A/B tests, and you can click on any one of them to dive deeper.

On a test’s stats page, you’ll see engagement graphs for both videos, as well as each video’s average engagement, play rate, total plays, and, if applicable in either video, how many actions (like someone clicking on a Call to Action) have occurred.

abtest-stats

Ending a Test

Once you’ve racked up enough stats that you feel you can make a meaningful decision between your two videos, you can end the test. To do that, just click that End Test button. A button will appear beneath each video to declare it the winner. Just click that button for the video you’ve deemed to be the victor, and we’ll automatically show that video to all visitors from then on!

“But Wistia,” you may ask, “why don’t you select the winner for me?” Every company’s video goals are different, and one A/B test may be looking for the video that has the highest play rate, while another is looking for the highest engagement. Or, you may just be considering the shape of the engagement graph to guide your choices. With so many possibilities for what winning entails, we’ve left it up to you so that you can get the most useful and actionable data out of your tests.

If you have any questions about the results you’re seeing, or need help with using the A/B Testing tool, don’t hesitate to contact us!