5 Lessons Learned from Tweet Testing

December 3, 2015

Jenny Coppola


I spend a lot of time tweeting. As Wistia’s social media coordinator, it’s the most tactical part of my job, from writing copy and scheduling tweets, to creating compelling images to accompany them — Twitter accounts for a pretty significant portion of my day.

When I joined Wistia a few months ago, I decided that we should be tweeting more. I changed our strategy from one that felt somewhat organic, to one that functioned like a well-oiled machine — two tweets per hour from 9 am – 6 pm, with about half of that frequency on weekends. After a couple of months, I noticed that our numbers were trending positively across the board. Success! Time to sit back, relax, and turn on autopilot.

Now, let’s throw that plan out the window.

Truth be told, that got boring, fast. One day after feeling particularly complacent and uninspired by our Twitter feed, I asked myself, "Could we be doing an even better job at this?" The answer was… well, maybe… probably. Yes.

So, I decided to run some tests over the course of two weeks to determine how frequency and timing of tweets affected our overall engagement on Twitter. The findings from these two weeks would then influence our strategy moving forward.


Week 1: The Normal Nancy

One tweet per hour from 9 am – 9 pm = 12 tweets per day. Compared to an average week, which was two tweets per hour from 9 am – 6 pm = 20 tweets per day.

After a week of tweeting only 12 times a day, our Twitter stats were trending downward in a number of areas, however not by much. The number of link clicks and retweets we received was only slightly lower than average, despite tweeting much less often. Our impressions dropped significantly from roughly 22.2K per day to a measly 15K.

Engagement, on the other hand, fared a bit better. We had more likes, replies, and a higher engagement rate than average, but yet again, not by much.

Week 2: The Insomniac

One tweet per hour, 24 hours a day = 24 tweets per day. Compared to an average week, which was two tweets per hour from 9 am – 6 pm = 20 tweets per day.

From a strictly quantitative standpoint, the Insomniac week blew the Normal Nancy week out of the water. We nearly doubled our average likes this week and saw a significant increase in link clicks and retweets. Our engagement rate, along with our average impressions per day, also increased. During the Insomniac week, we exceeded all of our normal weekly averages, except for one: replies.

3 retweets and 7 likes at 2:12 am. Who knew?

So, what does the fact that we got less replies this week tell us about the type of conversations we’re having on Twitter? When it comes to determining your tweet strategy, it’s important to ask yourself what your goals are, as they will likely affect how you define success.

What We Learned:

1. Tweeting constantly will get you nowhere.

The old saying “If you don’t have something nice to say, then don’t say anything at all,” applies to Twitter in a big way — just replace the word ’nice’ with ’valuable.’ You can spot an inauthentic Twitter account from a mile away. Queueing up several tweets an hour, every hour, two weeks in advance isn’t going to necessarily increase the number of likes, retweets, or replies you receive. In fact, it’ll probably do the opposite.

“The old saying “If you don’t have something nice to say, then don’t say anything at all,” applies to Twitter in a big way — just replace the word ’nice’ with ’valuable.’”

Monitoring real-time activity and being responsive is crucial. Knowing when not to post is a huge part of managing a social account, especially Twitter. We learned that by tweeting less frequently (one tweet per hour versus two) and by tweeting over a longer period of time, you can actually encourage engagement.

2. Don’t make too many assumptions about your audience.

Lots of folks like to use Twitter at work as a way to take a quick mental break, or to check in on what’s happening in the world. Or, at least that’s what I thought. As it turns out, some of the most unexpected times of day proved to be the most successful for engagement. It’s counterintuitive, but it’s true!

Out of our top 20 most-engaged with tweets during the Insomniac week (when we were tweeting 24 hours a day), 15 of them happened after 6 pm, many of them occurring between midnight and 2 am. One explanation for this could be that we have an international audience who’s active at that time, or perhaps some people just like to cruise Twitter during the wee hours of the night. Alternatively, it might have to do with what other people are doing (or aren’t doing) at that time. If there’s less tweet-noise at 2 am, there’s a greater chance your tweet will be seen and interacted with.

3. So many variables – it’s not an exact science.

Someone will always be shouting a little bit louder than you on Twitter. It’s just the nature of the beast. On any given day, world events from politics to pop-culture can literally take over the space, so be lenient and set realistic expectations. While there are always factors you can’t control, there are a few aspects that I, personally, could have better managed.

During the first week of testing, we held our TweetChat, #WistiaChat, which drastically affected our weekly averages. Our stats were tainted. Luckily, I was able to manually delete these tweets from our data, recalculate all of our averages, and subtract any likes or link clicks garnered during that period. If I could go back in time, I would’ve picked a different week altogether to save some time and effort. Can you say #lessonlearned? I can.

4. Be wary of your stats.

Awesome, you got a bunch of likes on a hilarious .gif! While this might look great on your weekly social report, what did you really accomplish with that tweet?

What type of content are you promoting? This is another key factor that could affect your numbers. If you plan to run a test, try to stick with the same few pieces of content for that period, otherwise your data could be skewed based on the popularity or accessibility of the content itself.

Twitter is a highly sensitive space. Sprinkle in a popular hashtag a few times throughout your tweets and suddenly your total impressions are through the roof! This puts a pretty meaningless spike in your data. Much like with lead scoring, you have to place a value on these interactions to understand whether they are actually qualified or not.

5. Think about the state of your audience today.

Every great brand on Twitter tailors their content to the needs of their audience. These needs often vary depending on the state of that audience. If your brand has an established presence online with thousands or even millions of followers, then your goals are going to look drastically different than those of a brand that just hit their 200 follower mark.

The bottom line is that there’s no one-size-fits-all measure for success on Twitter. I was under the impression that changing how often we tweeted and when would have a dramatic effect on engagement. And the truth is, it really didn’t!

Our engagement rates consistently hovered between 0.9–1.1 % over the course of these tests regardless of what we did. If the simple fact is that those two metrics don’t really affect engagement, then at least now we can cross them off our list and try to determine what will.

Now what? Keep testing.

Running tests is like eating chips — you can’t have just one. Our next set of experiments will look at how language (straight-forward versus flowery) affects overall engagement. What are you waiting for? You’ve got nothing to lose. Start experimenting with Twitter!

December 3, 2015

Jenny Coppola


Mailing list sign-up form

Sign up to get Wistia’s best and freshest content.

More of a social being? We’re also on Instagram and Twitter.