Two Thumbnails Up

Last Monday, we posted about optimizing your play rates, and in that post, we tested play rates for a video based on the thumbnail (so meta!). We know you’ve been shivering with anticipation all week, so without further ado, here are the results we saw from that test, which used two thumbnails and employed two separate tests for views from email and other views. We anticipated that the custom thumbnail with Jeff would perform better:

Views from Email: Test Results

• Dots / Automatic Thumbnail = 29.53% click to open ratio
• Jeff / Custom Thumbnail = 28.55% click to open ratio

Views On the Blog: Test Results

• Dots / Automatic Thumbnail = 27% play rate, 77% engagement
• Jeff / Custom Thumbnail = 26% play rate, 82% engagement

As many commenters called us out on, this test on our blog wasn’t a great one because:

• a.) The post and email suggested that the content was about testing play rates.
• b.) We have a pretty loyal audience with relatively clear expectations of the type of videos they are going to get in our email snacks.

So, this test didn’t really teach us anything. Oh well, we still got to make a fun video.

But luckily, we had a backup plan: we ran a second test at the same time because of our concerns about these factors.

The Real Test: Our Pricing Video

We tested 3 variants of thumbnails of the video on top of our pricing page. Here, there was no indication that a test was going on, and the audience mostly consisted of people relatively unfamiliar with Wistia, checking out our pricing plans as they considered whether to open an account.

The first thumbnail was a random frame of the video chosen automatically by the system. In this case, it was a pretty ugly screenshot of the Account Dashboard.

The second thumbnail was a frame of the video that we chose ourselves from within the Wistia app. We picked a frame that showed Jeff speaking because our previous testing has shown good results when the thumbnail has people in it.

The final thumbnail we made in Photoshop by adding some text to a frame grab and then uploading this customized image as the thumbnail.

Results of the Pricing Page Test

The Photoshopped thumbnail (hands) and the thumbnail we selected (Jeff) had similar play rates and a 35% improvement in play rate compared to the system-generated one. These two versions also had higher engagement rates than the one with an automatic thumbnail. Total result was an increase in total time watched of 50%.

What methods have you found most effective for improving the play rates of your videos?