Category: ActionScript

OSMF getBitrateForIndex bug

At Dreamsocket, we have been working on an OSMF based video player for one of our clients. An important aspect of the player is that it provides visual QOS (quality of service) information to the user about what is going on. For example, what the current stream bitrate is, the buffer length, frame rate, and other metrics are shown on screen so a user can self diagnose what is going on. If a customer is having issues and calls customer service, this information can be very helpful.

The Problem

Unfortunately, in using OSMF, we were constantly seeing incorrect bitrates being reported for the current stream. It was obvious that it was playing a 1400kbps stream when OSMF was telling us that it was a 300kbps stream. If you have ever worked with video at a low level in Flash, you will know that you can never expect events to fire exactly how you want. That turned out to be the case here. OSMF internally was looking only for NetStream.Play.TransitionComplete code being fired in for a onPlayStatus event. Well good luck with that. You can simply do a seek at the start of a video right before it does a transition, or do a few seeks while it is transitioning and you’ll never see that get fired off.

The Solution

Although the onPlayStatus handler doesn’t always fire, a NetStatusEvent event with a NetStream.Play.Start code does fire when a new bitrate begins. Therefore if you introspect the details property (which gives you the stream name) of the info object in that event and the stream is different than the current bitrate, you can concur that the bitrate has switched. One quirk here is that in some cases a NetStream.Play.Start can fire with event.info.details being null, so you have to watch out for that.

So how can you apply this to OSMF? Given OSMF is so black boxed, the error has to be fixed in a core class. To make it simple, we only touched a single function onNetStatus in org.osmf.net.NetStreamDynamicStreamTrait.

we added a var definition at the top of the function

var index:int;

we added an additional case statement

case NetStreamCodes.NETSTREAM_PLAY_START:
	if(event.info.details)
	{  // don't try to grab the index if there is no stream name
		index = dsResource.indexFromName(event.info.details);

                // if starting a stream that is not the current index, let the system know that it is transitioning
		if(index != this.currentIndex)
		{
			inSetSwitching = true;
			setSwitching(true, index);
			inSetSwitching = false;
		}
                // notify the system that the stream has transitioned
		setSwitching(false, index);
	}
	break;

within the transition start statement, we put a lock on the system to only indicate a stream was switching if the index was changing

	index = dsResource.indexFromName(event.info.details);
	if(index != this.currentIndex)
		setSwitching(true, index);

So there you go, correct bitrate reporting with OSMF. Hopefully someone on the team will see this and they fix the issue in the core vs us having to mod their code. Hope this helps someone.

[UPDATE 2-16]
To be clear, this really is a bug with the Flash Player and FMS. The Flash Player itself is not firing the onPlayStatus event with a NetStream.Play.TransitionComplete code. This would effect any non OSMF player as well that relied only on that code to determine what stream was switched to. The bug with OSMF is that it just doesn’t account for this quirk. Also, this happens in all OSMF versions that I’m aware of. We were using 1.5.

Simplifying StageVideo with StageVideoProxy

Adobe recently took a large step in changing the way video gets rendered in the Flash player. For the most part the player has always taken a large role in rendering video to the screen. With the release of Flash player 10.2 this has all changed. Now if you explicitly tell the player to, you can offload the video completely to the hardware which decreases CPU usage (literally to 0%), lowers memory usage, enables higher frame rates, and overall enables greater pixel fidelity and video quality.

Obviously with all the upcoming phones, tablets, and connected TVs this has an even higher impact on a video applications experience. At Dreamsocket we’ve been fortunate enough to be an early adopter with projects on these platforms. From developing video player’s tailored for Android phones to creating Cartoon Network and Adult Swim’s GoogleTV apps, we’ve been able to see the impact this new change has, in addition to figuring out the best way to integrate it into existing applications.

All that said, the one thing that really struck us was how unique the new StageVideo API is. I say unique because it is a completely new API and is completely different than working with the Video object. I understand the differences, however do you really need to force someone to go back and recode everything just to use it? I know it forces a user to explicitly think and spend time putting StageVideo into an app, but why? It should be EASY. I mean really EASY.

Enter StageVideoProxy. We wanted to go into all of our video applications and simply swap out a Video object reference for a new Class that would use StageVideo if present and fallback to using a standard Video object if it failed. By having a simple class that extended Video, we could just swap out the instance and proxy all the calls to StageVideo when it was available. This made it EXTREMELY easy to retro fit any existing application with StageVideo functionality. In addition, we kept all 10.2 API references in places that would not get called in players below 10.2, making the code backwards compatible.

Super right? Well we did find a few limitations. First, StageVideo doesn’t really have a way to clear the video, so we had set the size to 0 width and 0 height to “clear” it. Second, we don’t really know depth of display objects vs other display objects easily, so we don’t have a way to determine z depth for the videos. The second step doesn’t really matter though, given the implementation currently only supports a single StageVideo instance. I’d like to add support for multi videos, but I’m still unclear on how many I can have and what dictates that. I understand how to know when they are there and when you hit your limit, but I’d still like to fully understand it before I add it in.

Regardless, this is super useful code so merry nerdmas! Here you go in all its MIT licensed glory

/**
* Copyright (c) 2010 Dreamsocket Incorporated.
* All rights reserved.
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
**/


package com.dreamsocket.media 
{
	import flash.events.Event;
	import flash.events.StageVideoEvent;
	import flash.events.StageVideoAvailabilityEvent;
	import flash.events.VideoEvent;
	import flash.geom.Rectangle;
	import flash.media.StageVideo;
	import flash.media.StageVideoAvailability;
	import flash.media.Video;
	import flash.net.NetStream;
	
	public class StageVideoProxy extends Video
	{
		protected var m_netStream:NetStream;
		protected var m_stageVideo:Object;
		
		
		public function StageVideoProxy (p_width:int = 320, p_height:int = 240)
		{
			super(p_width, p_height);

			this.addEventListener(Event.ADDED_TO_STAGE, this.onAddedToStage);
			this.addEventListener(Event.REMOVED_FROM_STAGE, this.onRemovedFromStage);
		}
	
	
		override public function set height(p_value:Number):void
		{
			if(p_value != this.height)
			{
				super.height = p_value;
				this.layoutView();
			}
		}
		
		
		override public function get videoHeight():int
		{
			return this.m_stageVideo ? this.m_stageVideo.videoHeight : super.videoHeight;
		}
		
				
		override public function get videoWidth():int
		{
			return this.m_stageVideo ? this.m_stageVideo.videoWidth : super.videoWidth;
		}
	
			
		override public function set width(p_value:Number):void
		{
			if(p_value != this.width)
			{
				super.width = p_value;
				this.layoutView();
			}
		}
		
				
		override public function set x(p_value:Number):void
		{
			if(p_value != this.x)
			{
				super.x = p_value;
				this.layoutView();
			}
		}
		
		
		override public function set y(p_value:Number):void
		{
			if(p_value != this.y)
			{
				super.y = p_value;
				this.layoutView();
			}
		}

		
		override public function attachNetStream(p_stream:NetStream):void
		{
			if(p_stream != this.m_netStream)
			{
				this.m_netStream = p_stream;
				this.teardownStream();
				this.setupStageVideo();
			}
		}
		
		protected function setupSpriteVideo():void
		{		
			this.m_stageVideo = null;
			super.attachNetStream(this.m_netStream);		
		}
		
		
		protected function setupStageVideo():void
		{	// only setup the view when video is on stage and there is a netstream attached
			// this helps prevent as much as possible the time when a StageVideo is initialized
			if(!this.stage || !this.m_netStream) return;
			
			try
			{
				if(!this.m_stageVideo && this.stage.stageVideos.length >= 1)
				{
					this.m_stageVideo = this.stage.stageVideos[0];
					this.m_stageVideo.addEventListener(StageVideoEvent.RENDER_STATE, this.onRenderStateChanged);
					this.layoutView();
				}
				
				if(this.m_stageVideo)
				{
					this.m_stageVideo.attachNetStream(this.m_netStream);
				}
				else
				{
					this.setupSpriteVideo();
				}
			}
			catch(error:Error)
			{
				this.setupSpriteVideo();
			}
		}
		
		
		protected function teardownStream():void
		{	
			try
			{
				if(this.m_stageVideo)
				{
					this.m_stageVideo.viewPort = new Rectangle(this.x, this.y, 0, 0);
					this.m_stageVideo.attachNetStream(null);
				}
				else if(this.m_netStream)
				{
					super.attachNetStream(null);
					this.clear();
				}
				
			}
			catch(error:Error){}
		}	
		
			
		protected function layoutView():void
		{
			if(this.m_stageVideo)
				this.m_stageVideo.viewPort = new Rectangle(this.x, this.y, this.width, this.height);
		}

		
		protected function onAddedToStage(p_event:Event):void
		{
			//this.stage.addEventListener(StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY, this.onStageVideoAvailabilityChanged);
			this.setupStageVideo();
			this.layoutView();
		}
		
		
		protected function onRemovedFromStage(p_event:Event):void
		{	
			//this.stage.removeEventListener(StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY, this.onStageVideoAvailabilityChanged);
			this.teardownStream();
		}
		
				
		protected function onRenderStateChanged(p_event:Event):void
		{
			switch(StageVideoEvent(p_event).status)
			{
				case VideoEvent.RENDER_STATUS_UNAVAILABLE: 
					this.teardownStream();
					this.setupStageVideo();
					break;
			}
		}
		
		
		protected function onStageVideoAvailabilityChanged(p_event:Event):void
		{
			this.teardownStream();
			this.setupStageVideo();
		}
	}
}

Embedding JavaScript into a SWF

Developers that have been using Flex should be pretty familiar by now with the Embed metadata tag which allows you to embed assets into a SWF (like images, SWFs, XML, etc). This option has also been added to CS4 (CS4 uses the Flex SDK to complete this task).

What some of you may not know or though of, is that you can actually embed JavaScript libraries into your SWF and have your SWF write those libs to the pages DOM. We’ve been using this technique for a while. However, when someone asked for a reference on the subject, I searched the web and didn’t find one. So here we are :) .

This technique is quite simple. The basics of it are that you embed the JavaScript library using the embed syntax, then create a Class that references it when instantiated, instantiate that class, get the string representation of the instance, and send that string to the page to be embedded using a JavaScript eval statement.

To illustrate this, lets create a JavaScript file called hello.js which has a single function hello that throws an JS alert.

Hello.js

function hello()
{
     alert("hello");
}

Now lets create the ActionScript file that embeds the JS into a SWF, writes it to the page, and calls the hello function in the lib.

package 
{
	import flash.display.Sprite;
	import flash.external.ExternalInterface;

	public class EmbeddedJavaScriptExample extends Sprite
	{
		// embed the JavaScript into the SWF
		[Embed(source="hello.js", mimeType="application/octet-stream")]
		// create a class that can instantiate the JavaScript for embedding
		private static const HelloJS:Class;
		
		public function EmbeddedJavaScriptExample()
		{
			if (ExternalInterface.available)
			{
				// embed the JavaScript to the page
				ExternalInterface.call("eval", new HelloJS().toString());
				
				// the embedded JavaScript has a function call named hello
				// now that it has been embedded to the page call it
				ExternalInterface.call("hello");
			}
		}
	}
}

You can download the full example here.

Pretty nifty eh? Hope this shows you a neat little trick to use in distributing some of your SWF/JS libraries.

Media Framework license and pricing changes

Today we introduced a new license and pricing model for the Dreamsocket Media Framework.

The new license model:

  • $99 – Individual
  • $350 – Small Business (2 – 20 employees)
  • $895 – Midsize Business (21 -50 employees)
  • $1895 – Large Business (51+ employees)

To see all the details of the change and why we did it, check the post over on our site.

If you haven’t checked out or grabbed your copy of the product, now is a great time!

Introducing the Dreamsocket Media Player

Today we released the first “face/player” to our framework today, which you are free to throw on your site and use yourself. Check out below, go read the article we wrote up about it, and download it for yourself today!