When Wistia began, Flash video ruled the world. It wasn’t just the best way to stream video online — it was the only way. Eventually, the HTML5 video standard was conceived by the free web, and though it was young and imperfect, the community understood that it would one day be king.
At Wistia, we’ve been falling back to HTML5 for as long as it’s been around. There have been wars over codecs, conflicting implementations, and bugs along the way. But after years of development, the technology has finally improved to a point where we can use it as the basis of our default player.
This post exists to give clarity on the technical details of our new HTML5 player, Romulus, and the logic behind them.
I’m glad you asked! Here are some key events in HTML5 video land.
- 2008–2009: Chrome, Safari, and Firefox get HTML5 video support early on.
- March 2011: Internet Explorer 9 adds support for HTML5 video, marking the dawn of modern IE.
- June 2011: Chrome supports both webm and h.264. Says it will drop h.264 support, but never does.
- 2011–2012: There is much gnashing of teeth over webm vs. h.264.
- April 2013: Mozilla relents and allows h.264 support for Firefox, making it the last major browser to do so.
- April-October 2013: Chrome, Safari, and Firefox implement early drafts of the HTML5 fullscreen API.
- November 2013: Cisco purchases rights to h.264 and open sources it.
- November 2013: IE11 adds HTML5 fullscreen API support, the last major browser to do so.
The three major requirements for Romulus are HTML5 video, h.264 decoding, and fullscreen API support. On top of those requirements, browsers are becoming generally more hostile to Flash. For example:
- Safari no longer comes with Flash installed.
- Tablet versions of modern IE no longer allow Flash.
- Firefox now requires “click-to-flash” by default.
Looking at the circumstances as a whole, it’s pretty clear that now is the time to start the move to HTML5-by-default.
The move to a new HTML5 player enabled us to improve a lot of things. Enjoy this long and rambling list!
We now support frame by frame seeking. If you pause the video and hover over it with your mouse, you can use the arrow keys to step frame by frame. This improvement also allows smoother stream switching when switching to HD. Finally, because stream switching is built-in to HTML5 video, playlists can transition between videos without leaving fullscreen. I’ve been waiting for that one for a long time.
Using the Player API, you can change the playback rate of the video. One caveat: it isn’t in the documentation yet since Flash doesn’t support it at all!
We use the HTML5 fullscreen API to support plugins and Labs in fullscreen. However, we still only do it where it makes sense. The fullscreen API isn’t all-powerful. For example, Safari doesn’t allow any keyboard input in fullscreen mode.
A minor change that we really like: you can finally fullscreen before play.
Some browsers will save bandwidth by throttling. For example, Google Chrome tries to only buffer 25 seconds ahead of the current frame. We don’t control this feature, but it’s a cool bonus that comes with HTML5!
The intersection of Flash, graphics drivers, and the browser makes for some pretty gnarly bugs. HTML5 video cuts out one party, so there are fewer weird interactions. By simplifying, we improved the chances that your video will just work.
Flash had a lot of weird issues.
To be fair, it’s not entirely Adobe’s fault. Flash had to dance around third party drivers, operating systems, and different hardware. Each browser needed to maintain its own Flash integration. It’s impossible to account for all the possible permutations, and it’s impressive that Flash integrates as well as it does.
But in the end, people don’t care about that. They just want their videos to work. Here are some of the most aggravating bugs HTML5 will help us avoid:
I’m not sure how it gets into this weird state, but when it happens, the only fix is to restart your browser or computer. That’s not a good experience for anyone.
Modern browsers have been implementing CSS transforms, which frequently use hardware acceleration to speed up computation. But when you combine Flash and hardware-accelerated CSS transforms, you get strange and unpredictable results. The video may appear 600px below its real location, or get rendered upside down and backwards. Seriously.
A lot of these situations occur when people are building dynamic sites with sliders or custom popups. We expect it will be way easier to do cool things like that with Romulus!
On the whole, HTML5 is a huge upgrade, but there are some minor inconveniences that we want to work out.
IFrame embeds require “allowfullscreen” attributes for full Romulus support. There’s not exactly anything wrong with that — it’s a good security feature — but a lot of content management systems are built with whitelist HTML validation, and all but the newest validators don’t include those attributes. Twitter cards have a similar problem: we just provide the IFrame src attribute, and they don’t include “allowfullscreen” attributes by default. This situation should improve with time and familiarity.
A bigger issue is that our HTML5 assets are encoded in an MP4 container, and the MP4 format requires that you download a full MOOV atom before the video can be played. The MOOV atom’s size increases with the video’s length, which means a longer wait after clicking play for long videos.
Buffering immediately is a good way to lose a viewer, which is why we only use Romulus for videos under 15 minutes by default. In the meantime, we’re looking for different ways to get around this limitation.
Last, and hopefully least, some browsers, like Firefox on Windows, rely on the h.264 decoding implementation that’s native to the operating system. Whether that implementation works correctly 100% of the time is yet to be seen. We’re keeping our eye on it just in case. It may be that they start using Cisco’s h.264 implementation instead, which should normalize behavior across computers.
Flash is much maligned, but it still has its uses in the foreseeable future.
- It’s the best (and only) delivery method for browsers without HTML5 video. I’m looking at you, IE8 and below. Also you browsers that have neglected your updates for years. Grr.
- The FLV format — that’s what Flash uses instead of MP4 — handles long videos well because it doesn’t need to download all the metadata before it can play.
- Security? Flash doesn’t care about “allowfullscreen” attributes, so you can still fullscreen from those old (and possibly mangled) IFrame embeds.
- Also, it pains me to say it, but there are still some environments that are Flash-only. No IFrames, no scripts, just object/embed tags. I’m looking at you, Facebook! Get with the times!
I’ve mentioned in a few places that we are trying to intelligently choose which player, HTML5 or Flash, is used for each embed. I want to give more clarity on exactly how we plan to do that.
First, if you are familiar with our Player API, you might have seen an option called “platformPreference,” which allowed you to choose between “flash” and “html5” — that is, Flash and the native HTML5 player, not Romulus.
We are deprecating that option in favor of one called “playerPreference,” which allows you to choose between “auto,” “html5,” and “flash.” The difference is that, in this scheme, “html5” means Romulus if it’s supported, and the native HTML5 player if it’s not (notably IE9 and IE10). NOTE: “platformPreference” is deprecated, but still works and technically overrides “playerPreference.”
Here are browsers that, at time of posting, have basic Romulus support:
- Google Chrome
- Firefox on Windows
For full Romulus support, the video needs to meet the following requirements:
- It’s less than 15 minutes.
- It has high quality MP4 encodes. All new videos have these, but old ones probably don’t.
- It can be fullscreened. (If it’s in an IFrame, it must have “allowfullscreen” attributes.)
By default, “playerPreference” is set to “auto,” which means it will try to use Romulus when it detects full support. If it does not detect support, it will fall back to Flash, then native HTML5.
You can choose to manually set playerPreference to “html5” in your embed code. If you do that, it will bypass the additional checks for full support, using Romulus if it detects just basic support. If it does not detect Romulus support, it will fall back to native HTML5, then Flash.
We were careful and took our time with this launch. The plan was to enable Romulus slowly for interested accounts. Then, we’d gradually ramp up the user base, catch any major bugs, and release to everyone once it was polished.
We turned Romulus on for our own account in October 2013. For the next few months, as we talked to people in support, we enabled Romulus for those we thought could benefit from it. This turned out to be a good way to prove that Romulus would help solve some of the annoying issues we had with Flash.
After a while, we reached out to customers with large user bases and turned on Romulus for them. After a couple more months of small bug fixes, we figured out the logic for our “auto” player preference. That logic reduced our risk of deploying, so we decided it was time to accelerate the launch.
Two weeks before launch, we switched all new accounts to use Romulus by default. This let us test Romulus in the wild on a wide range of relatively small, low-risk accounts. Seeing no major issues, we continued on.
We were confident, so starting a week before launch, we enabled Romulus for a large chunk of our customers each day. All the accounts got up-to-date quickly, but we still had some leeway to address issues without affecting our entire user base.
There were a few small snags — for example, fullscreen mid-roll Turnstile had to be canned due to Safari’s restrictions on keyboard input — but on the whole, they didn’t affect playback and were easy to fix.
We’ve put a lot of thought into this change. With any luck, the current trends will continue. Browser technology will improve, users will upgrade to get HTML5 video support, and all will be right with the world.
Then, maybe one day, somewhere in the future, we can retire our old workhorse, Flash. It deserves a break.