W3C WebTV: Adaptive Streaming & Content Protection
Last week, the W3C held its Second Web & TV Workshop in Berlin. The workshop focused on the convergence of web technology and broadcasting. In other words, how will web and television work together to eventually merge?
Along with sessions on second-screen scenarios and accessibility, the workshops covered adaptive streaming and content protection. Both sessions were very compelling considering that streaming and protection are two important limitations of today's HTML5 video support.
Adaptive Streaming: DASH
Adaptive streaming is a technology that enables high-quality video streaming from any regular web server. Each adaptive stream is stored in multiple quality levels. Video players continuously request small fragments from these files (e.g. through range-requests) and seamlessly glue them together into one presentation. This technology is especially well suited for mobile (3G, 4G, WiFi) video delivery because video players can quickly adapt to changing bandwidth conditions by loading fragments from another quality level.
In the workshop's adaptive streaming session, the main focus was on MPEG DASH, a just-released specification from the Motion Pictures Experts Group. DASH aims to standardize streaming of video over HTTP, since today's solutions from Apple, Adobe and Microsoft are 95% the same but 100% incompatible.
In a nutshell, DASH specifies the format of the XML file that lists the available quality levels (the manifest). The spec provides guidance around encoding video for adaptive streaming (mostly for MP4) and around stitching the fragments together in a video player. See this paper and these slides from MPEG DASH co-author Thomas Stockhammer for more information.
Moving forward, two outstanding issues must be resolved in order to make DASH a widely used standard. First, DASH should be cleared of patent claims (if there are any), so it can be used in free software (e.g. for WebM). Second, specifications should be written for connecting DASH to HTML5, so adaptive streams can load in a video tag. For example, simply through the @src attribute.
Content Protection: PIFF
Content protection is another hot topic for online video. Currently, HTML5 video provides no content protection mechanisms, while closed systems like Flash and Silverlight do. Again, there is no standard for this, forcing companies like Netflix to encode their content multiple times for various DRM systems. And since DRM (by nature) is hard to specify in an open format, standardization seems far away.
There is progress though, in the form of PIFF (Protected Interoperable File Format). PIFF is based upon the MP4 file format and specifies what encryption should be used and how it should be applied. The beauty of this format is that it solely focuses on a common encryption mechanism, while leaving the rights management part untouched. This allows content owners to encrypt and store their videos once, even when using multiple DRM systems. See this paper by John Simmons for a more in-depth explanation.
The separation of encryption and rights management also opens the door for scenarios in which only encryption (no rights management) is used. Such scenarios should appeal to both publishers (for basic protection or privacy reasons) and open browsers like Firefox and Chrome. Next step in this area is the investigation of a common decryption workflow, in either HTML5 or DASH.
More News Soon
In summary, the Second Web & TV Workshop was exciting and productive, but there is still work ahead. Recognizing this, the W3C started a Web+TV Interest Group, which will continue work on such things as investigating the legal state of MPEG DASH. Stay tuned for more updates down the road...
Comments
Thanks for the update on DASH and PIFF. While looking at Apple HLS and its related solutions from Adobe, Microsoft and CodeShop, I was struck by how artificially complicated all this video stuff was, given the similarities. And that HTML5 would have to address it.
Just a quick note that the slides link on DASH has some HTML at the end of the href, and a thank you for your work on an Adaptive Branch. Seeing this post encourages me to believe that if there's an easy migration from HLS to DASH on *nix-style OSes using similar command-line tools, that the JW player will likely support DASH in future if I pick it now as a platform for non-Mac HLS adaptive streaming and might want to support DASH+WebM in future instead.
(That said, I feel like it might be my responsibility as a content encoder to support both in future, and I wonder what a Windows Phone 8 pushed by Nokia will support for adaptive streaming... Perhaps I'll forever have to push content to YouTube just to cover all my bases.)
Submitted by Louis St-Amour on Sun, 2011-02-20 21:58.
After further browsing, I agree that the Qualcomm[1] and Netflix[2] papers are the most informative. I also found the following presentation from Samsung (Oct 2010) on various adaptive streaming technologies to be useful, specifically the last 5 pages, though they did gloss over some things, particularly client requirements and Adobe's solutions.
http://docs.google.com/viewer?a=v&q=cache:KCyO0B9QcWAJ:edu.tta.or.kr/sub3/down.php?No%3D69%26file%3D...
I was fascinated by the IRC conversion into minutes, the results of which can be found here:
Day 1 (mostly web): http://www.w3.org/2011/02/08-webtv-minutes.html
Day 2 (mostly TV): http://www.w3.org/2011/02/09-webtv-minutes.html
Overall, I'm amazed at how much Microsoft is pushing for this to be royalty-free, though I suppose they stand the most to gain, given their browser/OS dominance, ability to produce smarter DRM-supporting, adaptive-streaming clients for web and TV, and so on. Cisco meanwhile appeared to be there just because they don't want anything interfering with their VoIP golden goose (not realising that once this is baked into browsers, they can kiss proprietary hardware goodbye), while Apple and Adobe are conspicuously absent from these conversations.
I'm sure you'll stand to gain from Adobe's lack of participation, given their own DRM and streaming solutions, and I suppose all that's left is to wait and see if Apple decides to support MPEG-DASH in OS X Lion at WWDC this year. It's funny how much trouble goes into what are ultimately very simple problems that have obvious solutions -- which no one until now has been able to agree on. ;-)
I wonder who will be first to market with an OAuth-based DRM for adaptive-streaming video. Not to mention how it gets supported in podcasts, and Apple's lack of documentation for PCP DTDs to specify formats in Atom/RSS. (I can't believe I'm reading iTunes U docs from Jan 2011 to figure this out.) I wonder what Podcast Producer 3 and the new FCP holds in store for us from an encoding perspective. Could I realistically hope for WebM? :p
And where was Google in all this, given YouTube's #1 status for video search, ads, etc.?
Linked to in original blog post:
[1] Qualcomm: http://www.w3.org/2010/11/web-and-tv/papers/webtv2_submission_64.pdf
[2] Netflix: http://www.w3.org/2010/11/web-and-tv/papers/webtv2_submission_62.pdf
Submitted by Louis St-Amour on Sun, 2011-02-20 23:40.
Thanks for the hint on the links - I fixed them.
You're right; the various streaming solutions are each brought as a unique piece of technology, but in essence it's all the same. With the JW Player, we'll indeed start supporting them all, as we get requests from users. So far, the only thing getting traction beyond enterprise video streaming is Apple's HLS.
With Smooth Streaming, Microsoft has the most mature and solid solution. They also have the webservers to plug the streaming into (IIS). What MS doesn't have is client-side traction - Silverlight is getting no attention compared to Flash, HTML5 and iOS apps. It's a good move from them to push for one standard, so they can serve all client frameworks with their servers.
Apple is very much involved with the DASH specification - I suspect they pushed MPEG TS support in the DASH spec ;) (iOS 5 software update?). Adobe is indeed absent, but Chrome/Youtube/WebM are definitely looking into DASH as well. In turn, DASH specifically does not require one codec or video container. It works equally well with WebM.
Submitted by JeroenW on Mon, 2011-02-21 04:40.
Yeah, my interest in HLS stems from being able to run it off plain vanilla Apache (if I wanted), and the same goes for DASH. I'd rather not pay for special IIS CDNs if I can help it. The fact that Apple *requires* HLS for 3G support in apps just cements the deal, though I wish they'd bump up the 3G quality a touch, I suspect a hard-coded limit as with 20MB for iTunes downloads. [That's also, I think, why m.youtube.com looks so much better than the YouTube app over 3G, even if it takes a bit longer...]
And I'm excited to see WebM consideration, if for no other reason than to keep MPEG-LA honest. (I liked how at the end of Feb 9's minutes, they said "the only similarity between MPEG and MPEG-LA are the four letters!")
Microsoft's Silverlight as a brand/add-on is dead thanks to HTML5, so unless they integrate it for Windows Media 12 or Windows 8, I don't see it continuing much further. It's only being used these days for Windows Phone 7 apps, and that will eventually shift as Nokia and WP8 (running on actual Windows) gains steam. It's ironic that Microsoft does have the best software for doing interactive editing on par or exceeding Flash CS5, yet rather than focus entirely on HTML5 like Apple, they spent years developing a Flash competitor instead. I'm certain Flash CS6 will be HTML5-driven, if Adobe wants their editing tools to stay relevant--if they don't, Microsoft Expression Blend will surely eat their lunch in another year or two or somebody new will come along (it's not impossible!).
Adobe should just move up the stack a bit to more of a building-block/services approach on top of HTML5 and Flash. FMS should focus entirely on cross-platform media processing and Adobe Connect-style collaboration rather than proprietary streaming and incompatible DRM. Imagine a toolset that lets you build on and work with video and audio streams as easily as you share IRC chat or tweets... It'd be interesting to see if, beyond MPEG-DASH, people looked at audio/video chat in-browser next ... Maybe when FaceTime finally gets a push from Apple, with its email-address contact support? Something in me says we'll see it from Chrome before Safari, though, given Google Talk's web-based nature versus iChat's software one.
Ahh, all this innovation in the browser. Feels like 1999 all over again, except we actually get along...
Submitted by Louis St-Amour on Tue, 2011-02-22 04:56.
I read with humor above in JEROENW's comment he wrote "Silverlight is getting no attention compared to Flash.."
Really? Have you heard of this site called www.netflix.com ???? :) Come on man. Silverlight is the de facto content playback platform now.
BTW: http://www.riastats.com/ Not sure if you saw that. Silverlight is sitting at about 70% market penetration..and growing.
Christopher
Submitted by Christopher Levy on Mon, 2011-05-23 12:07.
Silverlight is indeed seeing next to no usage. Youtube, Hulu, Vimeo, Facebook, Brightcove, Yahoo, Blip are all using Flash and HTML5 for video delivery. In fact, I wouldn't know a second service next to Netflix that uses Silverlight. And be aware Netflix is not using Smooth Streaming, but their own hand-rolled streaming protocol.
Don't get me wrong, Silverlight is a great technology. It's just not being used. With reason: at 70% penetration, you are blocking the door for 1 in 3 visitors. The combination of Flash + HTML5 will give you 100% coverage - without having to do transcoding.
Submitted by JeroenW on Tue, 2011-05-24 03:39.
I agree the comments on silverlight vs flash, and flash is dominate in the UK for tv over web but silverlight has drm for live and adobe have been very slow here compared to playread
Submitted by Phil Sellick on Sun, 2011-07-17 11:48.
Post new comment