Tag: open media formats
HTML5 Video: Still not there yet, but making headway
by Pablo Schklowsky on October 10, 2011
Last year, we declared that HTML5 video was not quite there yet. Well, it's nearly 18 months since that post, so what's happened in the intervening time?
Let's start off with the positive: compared to 18 months ago, the browsers' implementations of the <video> tag have become much more stable. They also all have built-in support for pseudo-streaming (i.e. the ability to seek to an unbuffered position in the video file). The tools surrounding HTML5 video — streaming servers, transcoders, etc. — have also improved.
On the other hand, many of the issues we pointed to in our last post have not yet been resolved. HTML5 video is still "not quite there yet", but we'll take a look at the developments over the past year and make some predictions on where this is all headed.
Answering some open questions
In our last post on the subject, Jeroen mentioned three major issues which prevented HTML5 video from taking off. Let's see where these issues are today.
Codec support
The major question surrounding codecs revolves around whether open video codecs such as Google's WebM and Ogg Theora would overtake proprietary formats such as H.264. As before, this still comes down to browser and device support. Though the open-source browsers have decided on an open format — WebM — no current mass-market mobile device has hardware drivers which can decode VP8 (the video codec used in WebM).
On the desktop, the sides seem firmly entrenched. Apple has no interest in moving away from H.264, considering the investment they've made in the technology. Microsoft doesn't support WebM natively (users need to manually install the codecs). Firefox and Opera still have no desire (or, possibly, ability) to support H.264. And although Google announced in January that H.264 support would be removed from Chrome, more than 8 months later, there's been no indication when (or even if) this will happen.
The codec wars are still being fought full-steam. And the end result of the lack of a standard video format is that many publishers simply encode once — in H.264 — and rely on Flash for support in browsers that don't support it.
Streaming
We predicted that the browsers would choose to implement their own streaming mechanism to compete with Apple's HTTP Live Streaming (HLS) format (built into desktop and mobile Safari). The question would be what open standard would be chosen to build upon?
What happened instead is that this area was more or less overlooked. Safari Desktop and Mobile, and Android 3.0+ are still the only browsers with built-in support for an HTML5 streaming protocol. Publishers who require that their content be streamed (rather than simply downloaded) on any other device or browser must fall over to a Flash or Silverlight player.
Moreover, a common streaming format has still not been agreed upon. The browser vendors are reluctant to hitch their wagon to any one method of HTTP streaming — for technical reasons, and because they're waiting to see which formats users begin implementing themselves. Of course, this is something of a chicken-or-egg proposition.
Fullscreen
A draft standard for adding fullscreen support to browsers has been written up and more or less agreed upon. Safari has already implemented support for this in version 5.1. While no other browsers have implemented the necessary APIs, Chrome (which uses the same rendering engine as Safari, called Webkit) is probably the closest to making this a reality. Although Mozilla wrote the spec, there's currently no word on when Firefox will implement it Firefox has implemented fullscreen support in their nightly build, but the feature is disabled by default. In short, true fullscreen functionality is still lacking on most browsers.
It's worth mentioning that on most mobile phones (not tablets), HTML5 videos will always play in fullscreen mode. This is partially due to limitations in hardware, but it also makes sense from a usability perspective — if you're watching video on a web page on a 3.5-inch screen, do you really want to see anything other than the video itself?
What's coming up
So what can we expect within the next 6-12 months? We recently attended the Open Video Conference in New York, and rubbed shoulders with the people who are making HTML5 Video happen — browser engineers from Apple, Google, Mozilla and Opera were all in attendance (not to mention YouTube, Netflix, Internet Archive and many more). Everyone had a ton to bring to the table (Silvia Pfeiffer, who organized the conference's Open Media Developer track, has written a great summary of the work that went on). And while there's a lot of exciting technology being built, the core problems I mentioned earlier which are preventing HTML5 from becoming the de facto standard in web video (format compatibility and streaming) are still not being addressed in a unified way.
That being said, some of that new stuff is pretty cool, and may end up moving the needle. Let's take a look.
The <track> Element: Adding Text Elements to Media
The HTML5 <track> element is for adding timed textual metadata to a media element. This enables video players to add functionality to support subtitles, closed captions, chaptering and more. While none of the browsers currently support <track> natively, we got to see a cool JavaScript demo which added captioning functionality into a <video> tag.
Once this becomes a reality, text tracks will be an exciting new development in HTML5. It will help make video accessible to the seeing- and hearing-impaired, enable subtitles for foreign language speakers, and make video files indexable (and therefore searchable). All of these will pave the way for video and audio to become first-class citizens on the web.
MediaSource API: Direct access to video data
Google is working on a feature for Chrome, called the MediaSource API, which will allow JavaScript developers to manipulate video data at the lowest possible level (the byte stream). One application of this mechanism would be for video players to piece together small bits of video, serving as the basis for an adaptive streaming application.
This approach puts the onus on JavaScript video players, such as the JW Player, and on video content providers, to come up with their own solutions for streaming video in HTML5. While I'm sure there will be attempts at making this happen, the jury's still out on whether this method can become the basis for HTML5 video streaming, for the usual reasons — browser fragmentation, and the ever-present format wars (Chrome's implementation will only support WebM at first). My personal opinion is that until the browsers present a standard for HTTP streaming to developers and publishers, you'll see a couple of cool demos, maybe even an end-to-end solution (think Netflix or YouTube), but not an overall standard.
Conclusions
Yes, progress has been made. The tools are better (for example, the popular desktop encoder Miro Video Converter now includes support for WebM). The servers are better (Wowza Media Server and Adobe Flash Media Server (FMS) support both Apple HLS for HTML5 streaming and RTMP for Flash streaming). And the browsers are better, especially the mobile ones (many of the HTML5 video bugs in Android and iOS were fixed in versions 2.3 and 4.0, respectively).
So why did I say at the beginning of this article that HTML5 still isn't there yet? Why isn't Flash losing more ground, considering its own drawbacks (poor resource utilization, lack of support on iOS, to name a couple)? The simple answer is that Flash offers a safer, more uniform cross-platform environment for video developers and publishers, while HTML5 presents a dizzying array of browsers to test against, and competing formats to support. You still need both Flash and HTML5 to cover all of your bases, but most publishers are still choosing Flash as their primary video delivery platform for these reasons.
The JW Player will continue to stay close to the bleeding edge of HTML5 video, and we'll continue integrating both Flash and HTML5 together as seamlessly as possible. We're rooting for HTML5 and its promise of open standards, better accessibility and better performance. But it'll still be some time before HTML5 has everything it needs to take its place at the head of the table.
FOMS: Funny Name, Serious Talk about HTML5, Bitrate Switching, WebM, and WebSRT
by Zachary Ozer on October 08, 2010
Last week, in the midst of our normal support requests and software development, we found a way to squeeze in a little extra time to attend the Open Video Conference and associated meetings here in New York. Everyone here at LongTail is a strong believer in open media formats, but it is rare to see so many people who are as committed to it as we are - assembled in one place.
For most of us, the most exciting part of this whole week was the Foundations of Open Media Software Workshop, organized by Silvia Pheiffer, a member of Xiph.org and frequent contributor to the HTML5 media specification.
Bitrate Switching
Going into the workshop, we were especially interested in hearing everyone’s thoughts on adaptive bitrate switching. While this is one of the most popular features of our Flash player, HTML5 doesn’t offer any mechanism for handling this, so we can’t offer this functionality in our HTML5 player. Thankfully, the workshop included a brilliant mix of browser vendors (Philip Jägenstedt from Opera, Chris Blizzard from Mozilla), Codec designers (John Luther of WebM and Christopher “Monty” Montgomery, Gregory Maxwell,Timothy Terriberry, and Ralph Giles of Ogg Theora and Vorbis), and player developers (Steve Heffernan of VideoJS, Michael Dale of Kaltura).
Several viable alternatives were presented for bitrate switching, however, there are several technical challenges to contend with. First, there is a question as to whether to use several long files and simply switch between them using range requests, or to break the files up into thousands of short (~5 second) "chunks", and play them back sequentially. Ultimately, using several long files presents a big challenge, both in terms of time to start playback and live streaming. All current file formats would require that the index of switch points be loaded for all files before starting the video. This dramatically increases the amount of time required to begin playing back a video and makes live streaming impossible, since the index could not be dynamic. Additionally, while a new file format could be designed to handle this, it would be difficult to achieve widespread adoption.
This meant that a mechanism that used several manifest files and many small files seemed quite attractive. The browsers would simply load the manifest for the appropriate quality, and then load a different manifest if necessary. Furthermore, the browser could periodically check for additional entries in the manifest, effectively allowing for a livestream with DVR. The browser vendors noted that this could be very simple: they will need to implement a mechanism for synchronizing media playback over multiple files for accessibility (alternate audio tracks) regardless, so why not use this to simply stitch together video files, one after another? Finally, this mechanism closely mimics Apple's bitrate switching mechanism, so it has a proven track record, and the tools built to support it.
Unfortunately, the format discussion was so involved, it left little time to discuss how the browsers would choose to switch streams. A loose JS API was proposed for this, with the possibility that browsers would build in their own default logic. However, based on this feedback from the group, we will draft a specification for this, post it for feedback, and include it in the JW Player as soon as possible.
WebM
One of the other highlights was the community’s interest in WebM. Much of the video on the web today relies on the H.264 for encoding video. Unfortunately, H.264 cannot always be used without paying royalties to a licensing organization, MPEG LA. While the HTML5 video spec was being developed, Ogg Theora was the only realistic open source alternative to H.264. Unfortunately, it’s performance is not nearly as good as H.264. WebM is a project that was started up by Google after their acquisition of ON2, which offers a high-performance, royalty-free codec for video. For this reason, the participants were especially interested in the possibility of adopting it as the baseline codec for HTML5. In fact, our HTML5 player already supports it, as both Opera and Chrome have this functionality built into their production browser, and Firefox has it available in their new beta. The biggest concern was that Apple and Microsoft have both refused to include support, and while Adobe recently announced VP8 support, there have been no dates set.
WebSRT
Finally, there was a discussion over web accessibility, specifically subtitles and closed-captions. This is near and dear to our heart, as JW recognized the importance of this early on, and made sure to include accessibility functionality ever since. The new standard for subtitling and captioning on the web will likely be WebSRT, and we’ll be working closely with the drafters of that specification and browser vendors to include support as soon as possible.
As intellectually stimulating (and fun) as these workshops are, it’s important to us that we’re a part of these discussions so that we can continue to develop the most innovative products, and that those products are standards based. So keep your eyes peeled – we’ll be working hard to integrate these technologies into all of our products as the standards develop.