HTML5 Video: Still not there yet, but making headway

Last year, we declared that HTML5 video was not quite there yet. Well, it's nearly 18 months since that post, so what's happened in the intervening time?

Let's start off with the positive: compared to 18 months ago, the browsers' implementations of the <video> tag have become much more stable. They also all have built-in support for pseudo-streaming (i.e. the ability to seek to an unbuffered position in the video file). The tools surrounding HTML5 video — streaming servers, transcoders, etc. — have also improved.

On the other hand, many of the issues we pointed to in our last post have not yet been resolved. HTML5 video is still "not quite there yet", but we'll take a look at the developments over the past year and make some predictions on where this is all headed.

Answering some open questions

In our last post on the subject, Jeroen mentioned three major issues which prevented HTML5 video from taking off. Let's see where these issues are today.

Codec support

The major question surrounding codecs revolves around whether open video codecs such as Google's WebM and Ogg Theora would overtake proprietary formats such as H.264. As before, this still comes down to browser and device support. Though the open-source browsers have decided on an open format — WebM — no current mass-market mobile device has hardware drivers which can decode VP8 (the video codec used in WebM).

On the desktop, the sides seem firmly entrenched. Apple has no interest in moving away from H.264, considering the investment they've made in the technology. Microsoft doesn't support WebM natively (users need to manually install the codecs). Firefox and Opera still have no desire (or, possibly, ability) to support H.264. And although Google announced in January that H.264 support would be removed from Chrome, more than 8 months later, there's been no indication when (or even if) this will happen.

The codec wars are still being fought full-steam. And the end result of the lack of a standard video format is that many publishers simply encode once — in H.264 — and rely on Flash for support in browsers that don't support it.

Streaming

We predicted that the browsers would choose to implement their own streaming mechanism to compete with Apple's HTTP Live Streaming (HLS) format (built into desktop and mobile Safari). The question would be what open standard would be chosen to build upon?

What happened instead is that this area was more or less overlooked. Safari Desktop and Mobile, and Android 3.0+ are still the only browsers with built-in support for an HTML5 streaming protocol. Publishers who require that their content be streamed (rather than simply downloaded) on any other device or browser must fall over to a Flash or Silverlight player.

Moreover, a common streaming format has still not been agreed upon. The browser vendors are reluctant to hitch their wagon to any one method of HTTP streaming — for technical reasons, and because they're waiting to see which formats users begin implementing themselves. Of course, this is something of a chicken-or-egg proposition.

Fullscreen

A draft standard for adding fullscreen support to browsers has been written up and more or less agreed upon. Safari has already implemented support for this in version 5.1. While no other browsers have implemented the necessary APIs, Chrome (which uses the same rendering engine as Safari, called Webkit) is probably the closest to making this a reality. Although Mozilla wrote the spec, there's currently no word on when Firefox will implement it Firefox has implemented fullscreen support in their nightly build, but the feature is disabled by default. In short, true fullscreen functionality is still lacking on most browsers.

It's worth mentioning that on most mobile phones (not tablets), HTML5 videos will always play in fullscreen mode. This is partially due to limitations in hardware, but it also makes sense from a usability perspective — if you're watching video on a web page on a 3.5-inch screen, do you really want to see anything other than the video itself?

What's coming up

So what can we expect within the next 6-12 months? We recently attended the Open Video Conference in New York, and rubbed shoulders with the people who are making HTML5 Video happen — browser engineers from Apple, Google, Mozilla and Opera were all in attendance (not to mention YouTube, Netflix, Internet Archive and many more). Everyone had a ton to bring to the table (Silvia Pfeiffer, who organized the conference's Open Media Developer track, has written a great summary of the work that went on). And while there's a lot of exciting technology being built, the core problems I mentioned earlier which are preventing HTML5 from becoming the de facto standard in web video (format compatibility and streaming) are still not being addressed in a unified way.

That being said, some of that new stuff is pretty cool, and may end up moving the needle. Let's take a look.

The <track> Element: Adding Text Elements to Media

The HTML5 <track> element is for adding timed textual metadata to a media element. This enables video players to add functionality to support subtitles, closed captions, chaptering and more. While none of the browsers currently support <track> natively, we got to see a cool JavaScript demo which added captioning functionality into a <video> tag.

Once this becomes a reality, text tracks will be an exciting new development in HTML5. It will help make video accessible to the seeing- and hearing-impaired, enable subtitles for foreign language speakers, and make video files indexable (and therefore searchable). All of these will pave the way for video and audio to become first-class citizens on the web.

MediaSource API: Direct access to video data

Google is working on a feature for Chrome, called the MediaSource API, which will allow JavaScript developers to manipulate video data at the lowest possible level (the byte stream). One application of this mechanism would be for video players to piece together small bits of video, serving as the basis for an adaptive streaming application.

This approach puts the onus on JavaScript video players, such as the JW Player, and on video content providers, to come up with their own solutions for streaming video in HTML5. While I'm sure there will be attempts at making this happen, the jury's still out on whether this method can become the basis for HTML5 video streaming, for the usual reasons — browser fragmentation, and the ever-present format wars (Chrome's implementation will only support WebM at first). My personal opinion is that until the browsers present a standard for HTTP streaming to developers and publishers, you'll see a couple of cool demos, maybe even an end-to-end solution (think Netflix or YouTube), but not an overall standard.

Conclusions

Yes, progress has been made. The tools are better (for example, the popular desktop encoder Miro Video Converter now includes support for WebM). The servers are better (Wowza Media Server and Adobe Flash Media Server (FMS) support both Apple HLS for HTML5 streaming and RTMP for Flash streaming). And the browsers are better, especially the mobile ones (many of the HTML5 video bugs in Android and iOS were fixed in versions 2.3 and 4.0, respectively).

So why did I say at the beginning of this article that HTML5 still isn't there yet? Why isn't Flash losing more ground, considering its own drawbacks (poor resource utilization, lack of support on iOS, to name a couple)? The simple answer is that Flash offers a safer, more uniform cross-platform environment for video developers and publishers, while HTML5 presents a dizzying array of browsers to test against, and competing formats to support. You still need both Flash and HTML5 to cover all of your bases, but most publishers are still choosing Flash as their primary video delivery platform for these reasons.

The JW Player will continue to stay close to the bleeding edge of HTML5 video, and we'll continue integrating both Flash and HTML5 together as seamlessly as possible. We're rooting for HTML5 and its promise of open standards, better accessibility and better performance. But it'll still be some time before HTML5 has everything it needs to take its place at the head of the table.

Posted by Pablo Schklowsky on October 10, 2011