Tag: bitrate switching

What is Video Streaming?

by Jeroen Wijering on April 27, 2011

With all of the buzz around HTML5 and the iPad, there's been a lot of talk about the technologies underlying digital video. Besides the inevitable codecs (H264 & VP8), experts are discussing video delivery mechanisms, using indecipherable acronyms like RTMP, CDN and HLS. This blog post will give an overview of the various video streaming methods in plain English and bring the all-round developer and publisher up to date.

In a nutshell, there are three widely used ways to stream a video: Progressive Download, RTMP/RTSP Streaming, and Adaptive HTTP Streaming. We'll look at the three in detail here, describing their pros, cons, and various technologies that support each.

Note: more streaming methods (like MultiCast or Peer 2 Peer) exist, but are of little use to the regular publisher. They are not widely adopted and/or only useful in specific situations.

Progressive Download

Progressive Download is the most widely used video delivery method by far (in part because it's what YouTube uses). It's also easiest to implement: just put a video on your webserver and point your player to the URL. Once a user hits play, the player immediately starts downloading the file. The player will start video playback as soon as it has enough data to do so, but it will continue to download until it has received the whole file (hence the progressive).

Progressive Download is supported by Flash, HTML5 browsers, the iPad/iPhone and Android. On the server side, every regular webhoster supports downloads, as does every CDN (Content Delivery Network; webhosters that special in large-scale delivery). In most cases (Flash needs a small server module), it is possible to seek in a player to a not-yet-downloaded part of the video. At that point, the player re-downloads the video, starting at the seek offset instead of at the beginning. We call that feature pseudo-streaming.

The simplicity of Progressive Download also has its downsides. For one, bandwidth is wasted on data downloaded but not watched. Consider a user watching a ten minute video. They may leave the page after having watched only one minute of the video, but at that point the other nine minutes have already been downloaded. This means that the publisher has paid to transfer nine times as much data as the user actually watched - an expensive proposition on a large scale.

Another downside is the inability to change the quality of the video mid-stream: once the download starts, the video quality is locked. After switching a player to fullscreen, you generally see a blurry video, because it was intended to be watched at a much smaller size. Or, when you watch video on an iPad, your connection may switch from WiFi to 3G. Playback then stutters, because the download speeds are much lower on 3G.

In sum, Progressive Download works fine for short clips (a few minutes). For longer videos, the downsides start to impact playback too much. Plus, live streaming is not possible, as there's no downloadable file.

RTSP/RTMP Streaming

Because of the downsides of Progressive Download, RTMP/RTSP Streaming is widely used by professional media organizations like Hulu. This method uses specialized webservers that only deliver the frames of a video the user is currently watching. No data is downloaded in advance and data a user has seen is immediately discarded.

The most widely solution is used is RTMP (Real Time Messaging Protocol), the streaming protocol of Flash. It is supported by servers such as FMS and Wowza and most CDNs (but not by regular webhosters). HTML5 does not include a dedicated streaming protocol, nor does the iPad/iPhone. Android has support, for RTSP (Real Time Streaming Protocol). Unfortunately, RTSP is not widely supported by servers and CDNs.

This lack of support, especially on the server side, is the biggest drawback of RTMP/RTSP Streaming. Most publishers do not want to maintain expensive, dedicated servers to stream their videos. Additionally, the dedicated protocols (RTMP and RTSP) are often blocked by corporate firewalls.

On the plus side, RTMP streaming can change video quality mid-stream. This allows for optimal playback quality in the fullscreen and WiFi/3G scenarios described above. However, if the connection speed drops below the minimum bandwidth needed for the video, playback will be continuously interrupted. Unlike progressive download, users cannot pause a video and wait for enough data to download to ensure smooth playback.

In sum, RTMP/RTSP Streaming works great even for long-form or live video. It has specific server and protocol requirements, which makes it less accessible and adds significant complexity and cost as compared to Progressive Download.

Adaptive HTTP Streaming

Adaptive HTTP Streaming is a fairly new streaming format. It attempts to join the merits of RTMP/RTSP Streaming (bandwidth efficiency, quality switching) with those of Progressive Download (no special servers or protocol needed). Adaptive HTTP Streaming works by storing your videos on the server in small fragments (a few seconds each). The player then glues these fragments together into a continuous stream.

At present, Adaptive HTTP Streaming is supported by both Flash and the iPad/iPhone. Android supports it as of version 3 and support in HTML5 is currently under development. Since Adaptive HTTP Streaming leverages standard webservers, it is supported by webhosters and CDNs alike.

Although Adaptive HTTP Streaming eliminates many of the downsides of RTMP/RTSP Streaming and Progressive Download, it still has issues of its own, the biggest being the lack of standardization. Because it is a new technology, there is no single, widely used implementation. The most popular is currently Apple's HLS (HTTP Live Streaming), which is supported by the iPad/iPhone and Android 3.0. However, both Adobe and Microsoft have competing offerings (Zeri & Smooth) and the MPEG consortium is working on a standard named DASH.

It's also worth noting that none of the Adaptive HTTP Streaming implementations work with regular MP4 files. They all require your files to be converted from a regular MP4 into a specific fragmented format. Apple, Microsoft and Adobe each supply a tool for this, but support for these formats doesn't exist in regular video editors and transcoding tools (yet).

In summary, while Adaptive HTTP Streaming will likely become the single video streaming method over time, the technology is still fragmented (no pun intended) and ecosystem support is only beginning to arrive.

Summary: Streaming Support

This table sums up support for the various streaming methods across devices and servers.

Comments (19)   Tweet   Share

FOMS: Funny Name, Serious Talk about HTML5, Bitrate Switching, WebM, and WebSRT

by Zachary Ozer on October 08, 2010

Last week, in the midst of our normal support requests and software development, we found a way to squeeze in a little extra time to attend the Open Video Conference and associated meetings here in New York. Everyone here at LongTail is a strong believer in open media formats, but it is rare to see so many people who are as committed to it as we are - assembled in one place.

For most of us, the most exciting part of this whole week was the Foundations of Open Media Software Workshop, organized by Silvia Pheiffer, a member of Xiph.org and frequent contributor to the HTML5 media specification.

Bitrate Switching

Going into the workshop, we were especially interested in hearing everyone’s thoughts on adaptive bitrate switching. While this is one of the most popular features of our Flash player, HTML5 doesn’t offer any mechanism for handling this, so we can’t offer this functionality in our HTML5 player. Thankfully, the workshop included a brilliant mix of browser vendors (Philip Jägenstedt from Opera, Chris Blizzard from Mozilla), Codec designers (John Luther of WebM and Christopher “Monty” Montgomery, Gregory Maxwell,Timothy Terriberry, and Ralph Giles of Ogg Theora and Vorbis), and player developers (Steve Heffernan of VideoJS, Michael Dale of Kaltura).

Several viable alternatives were presented for bitrate switching, however, there are several technical challenges to contend with. First, there is a question as to whether to use several long files and simply switch between them using range requests, or to break the files up into thousands of short (~5 second) "chunks", and play them back sequentially. Ultimately, using several long files presents a big challenge, both in terms of time to start playback and live streaming. All current file formats would require that the index of switch points be loaded for all files before starting the video. This dramatically increases the amount of time required to begin playing back a video and makes live streaming impossible, since the index could not be dynamic. Additionally, while a new file format could be designed to handle this, it would be difficult to achieve widespread adoption.

This meant that a mechanism that used several manifest files and many small files seemed quite attractive. The browsers would simply load the manifest for the appropriate quality, and then load a different manifest if necessary. Furthermore, the browser could periodically check for additional entries in the manifest, effectively allowing for a livestream with DVR. The browser vendors noted that this could be very simple: they will need to implement a mechanism for synchronizing media playback over multiple files for accessibility (alternate audio tracks) regardless, so why not use this to simply stitch together video files, one after another? Finally, this mechanism closely mimics Apple's bitrate switching mechanism, so it has a proven track record, and the tools built to support it.

Unfortunately, the format discussion was so involved, it left little time to discuss how the browsers would choose to switch streams. A loose JS API was proposed for this, with the possibility that browsers would build in their own default logic. However, based on this feedback from the group, we will draft a specification for this, post it for feedback, and include it in the JW Player as soon as possible.

WebM

One of the other highlights was the community’s interest in WebM. Much of the video on the web today relies on the H.264 for encoding video. Unfortunately, H.264 cannot always be used without paying royalties to a licensing organization, MPEG LA. While the HTML5 video spec was being developed, Ogg Theora was the only realistic open source alternative to H.264. Unfortunately, it’s performance is not nearly as good as H.264. WebM is a project that was started up by Google after their acquisition of ON2, which offers a high-performance, royalty-free codec for video. For this reason, the participants were especially interested in the possibility of adopting it as the baseline codec for HTML5. In fact, our HTML5 player already supports it, as both Opera and Chrome have this functionality built into their production browser, and Firefox has it available in their new beta. The biggest concern was that Apple and Microsoft have both refused to include support, and while Adobe recently announced VP8 support, there have been no dates set.

WebSRT

Finally, there was a discussion over web accessibility, specifically subtitles and closed-captions. This is near and dear to our heart, as JW recognized the importance of this early on, and made sure to include accessibility functionality ever since. The new standard for subtitling and captioning on the web will likely be WebSRT, and we’ll be working closely with the drafters of that specification and browser vendors to include support as soon as possible.

As intellectually stimulating (and fun) as these workshops are, it’s important to us that we’re a part of these discussions so that we can continue to develop the most innovative products, and that those products are standards based. So keep your eyes peeled – we’ll be working hard to integrate these technologies into all of our products as the standards develop.

Comments (2)   Tweet   Share