Incredibly Inefficient MP4/H.264 RTMP Streaming

OK, false alarm, not a bug but an idiot end user.

I didn't realise that 4.x versions of the player default to high quality (smoothing and deblocking on [http://code.jeroenwijering.com/trac/wiki/FlashVars] - whatever that means).

Setting the 'quality' flashvar to "false" has improved things a great deal, in fact it's made a massive difference. Videos now play on lower spec. machines but still with very high CPU usage. Annoyingly because of the way that Flash works, on a lower spec. PC it appears to be almost twice as efficient to watch videos at full-screen than it does at the video player's native size. Clearly Flash + MP4/H.264 is exponentially more processor intensive than Flash + FLV - leaving the default high quality setting enabled doesn't matter as FLV streaming is so efficient within Flash.

In case others stumble across this: I'd already had a fiddle with the 'quality' Flash parameter (NOT flashvar) but this didn't seem to make much difference. However, it's still worth considering when embedding any Flash application. I've now left this parameter set to 'autolow' which I think is the best compromise (http://kb.adobe.com/selfservice/viewContent.do?externalId=tn_12701&sliceId=1).

@Peter,

I'm extremely surprised that you are seeing MP4/H.264 video using more CPU cycles that the equivalent FLV.

My experience is exactly the opposite.

Assuming that you are not re-sizing on the client (which always kills Flash), I usually see MP4/H.264 using 30-50% less CPU than the equivalent FLV.

I'm using MP4/H.264 encoded with ffmpeg 2-pass at 512kbps and 1024kbps, always in multiples of 16 for the width and height. This doesn't always result in exactly 16:9, but it's close enough. The MP4 is generally about 60% of the FLV and the quality is better.

Using correctly-sized MP4. has enabled me to run four players, one at 432x240 and three PIP at 114x64 with only about 30% CPU usage.

Good performance in fullscreen is very dependent on the availability of hardware acceleration. If it's not available, I would not even consider going to full screen. Same for high-bitrate video. Unless the client has a multi-megabit broadband connection, it simply doesn't work. Try playing one of the high-quality, high-bitrate YouTube videos (fmt=22) on a low-speed connection, it doesn't work, even though they have the best encoding on the Internet.

@kLink

Thanks for the response.

I'm not the person who encodes the videos so am uncertain of our exact method (I don't fully understand the whole concept of 'double pass' for example).

Assuming that you are not re-sizing on the client (which always kills Flash)

Are you saying that setting the video player size to anything other than the video's native resolution + control bar is a bad idea?

If so, then following is not good, is it?:

Video Player size = 640x380 (where 380 = 360+20) but videos being loaded into the player are either 768x432 or 1280x720 so we're actually reducing them down rather than blowing them up. Does that make a +ve or -ve difference?

I always thought it was good advice to have the video size 1/2 that of your player's 'native' view? Is this true/false?

A simple test will show you what the effect is:

Size a player's display area 768x432 and play the video. Open Task Manager on Windows or the equivalent on other OSs. Note the average CPU usage.

Size the player's display area 384x216 (half-size, but 1/4 the total area). Check the CPU usage.

Two pass encoding basically means that the encoder studies the original video and collects some information about it.
The collected information is then used for "look ahead to see what's going to be happening in the next bunch of frames", which makes the encoding more intelligent. Therefore the encoder can produce better compression and quality.

Using large video scaled down also increases the bandwidth requirement for real-time viewing and increaes the total traffic for viewing a video (much larger files).

Thanks for the 'double pass encoding' explanation, I've checked and we're definitely using this method.

I've given your suggestion a go, with the following results:

Video size: 768x432
Bit-rate: 384Kbps

PC1 (Dual-core PC)
Flash player size of 768x432(+20): CPU ~ 20-30%
Flash player size of 384x216(+20): CPU ~ 10-20%
Full-screen: CPU ~ 5-15%

PC2 (Single-core laptop)
Flash player size of 768x432(+20): CPU ~ 70-80%
Flash player size of 384x216(+20): CPU ~ 30-40%
Full-screen: CPU ~ 50%

Note there's another aspect to this issue: the compression. The more you squeeze the bitrate of a video, the more the CPU also has to work to rebuild the original stream again. So trying to get the video as small as possible sometimes results in a file that takes takes so much CPU power it can be hard to decode.

You can try this by encoding a video at a number of bitrates. Next to the degrading quality, you'll also see an increase in CPU at a certain point.

The last factor is the H.264 profile you use. Although the various HIGH profiles reslut in smaller filesizes, they are harder to decode on the client than the Baseline or MAIN profile. I recommend you use the Main profile. Some reading:

http://en.wikipedia.org/wiki/H.264#Profiles

Thanks, guys, this is *extremely* helpful.