Captionate - Receiving Captionate Embedded Data

All external FLV files are played using NetConnection and NetStream objects. If you have played video using a component, you may have not seen these objects, but internally all components also use these objects to play external FLV files.

FLV data events are called for the current NetStream object playing the FLV. For example, when the Flash Player encounters an onCaption event, it calls the .onCaption function of the NetStream object.

First, lets have a look at how external video file can be played using Flash and actionscript:

For Flash 8:

Create a new Flash document.
In the Library, choose new video command from the context menu. In the Video Properties dialog, make sure video (actionscript controlled) is selected.
Drag your newly created video symbol to stage.
Click on the video symbol on stage if necessary first and in the Property inspector, assign an instance name. For the following code to work, use the instance name: video.

Click on frame 1 on the timeline and enter the following actionscript (in Actions panel F9):

	nc = new NetConnection();
	nc.connect(null);
	nets = new NetStream(nc);
	video.attachVideo(nets);
	nets.play("test.flv");

Save the FLA file to a folder where you have a sample FLV file named test.flv.
When you Test Movie (Ctrl+Enter), your FLV will play.

For Flash MX 2004:

Create a new Flash document.
In the Library panel (Window > Library) select New Video from the Library options menu.
Drag your newly created video symbol to stage.
Click on the video symbol on stage if necessary first and in the Property inspector, assign an instance name. For the following code to work, use the instance name: video.

Click on frame 1 on the timeline and enter the following actionscript (in Actions panel F9):

	nc = new NetConnection();
	nc.connect(null);
	nets = new NetStream(nc);
	video.attachVideo(nets);
	nets.play("test.flv");

Save the FLA file to a folder where you have a sample FLV file named test.flv.
When you Test Movie (Ctrl+Enter), your FLV will play.

As you see what we do is this: first we create a video symbol with an instance name. Then, in the 5 lines of actionscript, we create a NetConnection object, then a NetStream object, then we attach the video symbol instance to the NetStream object. Then we supply the FLV name to the play function of the NetStream object and the FLV plays.

This sample is probably the shortest code to play a FLV file. In a real application you may want to do more, including providing a controller.

In order to receive any data there might be in the FLV file, we need to add event handlers to the NetStream object instance (nets in the above example) which has the video instance attached.

Adding the event handlers are quite simple, but you need to make sure you add the handler functions to the correct NetStream instance. For example, in a class code, while seems obvious, you need to make sure the NetStream object is created before adding the events.

Now lets add an onCaption event handler to our sample:

	nc = new NetConnection();
	nc.connect(null);
	nets = new NetStream(nc);
	video.attachVideo(nets);
	
	nets.onCaption = function(captions,speaker){
		trace('onCaption event at '+nets.time);	
	}
	
	nets.play("test.flv");

Also note that when using AS2, you may need to use the [""] syntax, as follows, which is exactly the same thing, written in a different way:

	nets["onCaption"] = function(captions,speaker){
		trace('onCaption event at '+nets.time);	
	}

Lets copy onCaption and onCaptionInfo handler samples from Captionate's help file, events section:

	nc = new NetConnection();
	nc.connect(null);
	nets = new NetStream(nc);
	video.attachVideo(nets);
	
	nets.onCaption = function(captions,speaker){
		trace('onCaption event at '+nets.time);
		trace(' Speaker index: '+speaker);
		for (n=0;n<captions.length;n++){
			str=captions[n];
			trace(' Caption for track '+n+' > "'+str+'"');
	}

	nets.onCaptionInfo = function(info){
		trace('onCaptionInfo event at '+nets.time);
		speakers=info.speakers;
		for (n=0;n<speakers.length;n++){
			obj=speakers[n];
			trace(' Speaker '+n+' > '+
			' name: "'+obj.name+'"'+
			' data: "'+obj.data+'"' );
		}
		tracks=info.tracks;
		for (n=0;n<tracks.length;n++){
			obj=tracks[n];
			trace(' Track '+n+' > '+
			' displayname: "'+obj.displayname+'"'+
			' type: "'+obj.type+'"'+
			' languagecode: "'+obj.languagecode+'"'+
			' targetwpm: '+obj.targetwpm+
			' data: "'+obj.data+'"' );
			}
		}

	}

	
	nets.play("test.flv");

Now our simple sample can trace all the data from onCaption and onCaptionInfo events. You can add other event handlers from Captionate's help and trace the data.

As you see every event has its own parameters and data structure. For example, onCaptionInfo has speakers and tracks arrays. onCaption has two parameters, one holds caption text for all language tracks and the other is index to the speakers array. The type of the parameter for onMarker is a string... So, you need to add the correct event handler to the NetStream instance and in the handler function, you need to 'parse' the data received...

'That's fine', you say, 'but I use the MediaPlayback component in Flash MX Professional 2004, now what?'

All components play the FLV internally similar to our sample above, using the NetStream object. You'll need to add the events to the NetStream instance in the components instance on stage, indeed very similar to our simple sample.

Flash MX Professional 2004 MediaDisplay Component
Lets say the instance name of the MediaDisplay component is md. The Netstream object instance can be reached as md._playerImpl._ns:.

	var nets = md._playerImpl._ns;

Using the above line, you can get the NetStream object instance to the nets variable and use Captionate help samples for adding events.

Flash MX Professional 2004 MediaPlayback Component
Lets say the instance name of the MediaPlayback component is mp. The Netstream object instance can be reached as mp._display._playerImpl._ns:.

	var nets = mp._display._playerImpl._ns;

Using the above line, again you can get the NetStream object instance to the nets variable and use Captionate help samples for adding events.

(If this is not clear: Create a new Flash document, drag a MediaPlayback component on stage, assign component instance name as 'mp'. In the component inspector, enter the path to your FLV into the URL field. Add the above code to the first frame, then append handler functions just as the first sample , i.e. like: nets.onCaption = function(captions,speaker){....).

Flash 8 Professional FLVPlayback Component
FLVPlayback unfortunately does not expose the internal NetStream object like other components, as it supports multiple videos. Still you can add events by replacing the original _createStream function.

Create a new FLA and drag a FLVPlayback component to stage. Then enter the following code to the first frame:

_global.mx.video.VideoPlayer.prototype._createStream  =
function () {
           //original _createStream code
           this._ns = new NetStream (this._ncMgr.getNetConnection());
           this._ns.mc = this;
           if (this._ncMgr.isRTMP()) {
               this._ns.onStatus = function (info) {
                   this.mc.rtmpOnStatus(info);
               };
           } else {
               this._ns.onStatus = function (info) {
                   this.mc.httpOnStatus(info);
               };
            }
           this._ns.onMetaData = function (info) {
               this.mc.onMetaData(info);
           };
           this._ns.onCuePoint = function (info) {
               this.mc.onCuePoint(info);
           };
           this._ns.setBufferTime(this._bufferTime);

           //new code
           this._ns.onCaption = function(captions,speaker){
             trace("onCaption");
          }
          // add other events here
       };

The above is really a hack-ish workaround. Converting captions to cue points will suit your needs most of the time. (Also please see the contribution by Peter Edwards below).

Handling the received data will be covered in another article.

Still, if you are not aware, the easiest way to display text on stage is using a dynamic text with a variable. Create a text field on stage using the text tool. In the property inspector, set the type to 'dynamic text' and input a variable name into the 'var' field. Now, anytime you assign to that variable, it will be displayed in the text field.

Lets see this in action!

Create a new FLA in Flash 8 and drag a FLVPlayback component on stage. Set the content path to your FLV.
In a new layer, draw a rectangle, convert it to symbol, change color to cyan (or white), alpha to 50% and put it over the FLVPlayback component.
In a new layer, create a text field using text tool, change type to 'dynamic', enter 'caption' as the 'var' name. Put it over the rectangle. Select 'Render as HTML' if your captions have HTML formatting, set the font.
Enter the code above for FLVPlayback component to the first frame. Add the following line to onCaption function:
```
	caption = captions[0];
```
This will display the caption text for the first language track, it will do for now.
Here's the result:
Hint: You may add a button someplace in your interface that toggles the visibility of the captions.

I thought you might like to know that your example for Flash 8 Professional FLVPlayback Component can be replaced with some much simpler, less hacky code:

// save the old function
var tmp_createStream : Function = _global.mx.video.VideoPlayer.prototype._createStream;

_global.mx.video.VideoPlayer.prototype._createStream = function() : Void
{
  // apply the old function in the instance context
  tmp_createStream.apply(this, null);

  // add our specifics
  this._ns.onCaption = function(captions,speaker) : Void
  {
    trace("onCaption");
  }
};

In fact, if you write a function like:

  private function fixVideoPlayer(func : Function) : Void
  {
    var tmp : Function = _global.mx.video.VideoPlayer.prototype._createStream;

    _global.mx.video.VideoPlayer.prototype._createStream = function() : Void
    {
      tmp.apply(this, null);
      this._ns.onCaption = func;
    };
  }

Then just before the instance of the FLVPlayback component is created, you can make a call like:

fixVideoPlayer(Delegate.create(this, this.onCaption));

and define the onCaption function in your class appropriately. This could even work per instance / per VideoPlayer with a little tweaking.

An even Better Version:
by Peter Edwards of derivco.com (July 18, 2006)

function fixVideoPlayerInst(player : mx.video.VideoPlayer, nsFuncName:
String, func : Function) : Void
{
  var tmp : Function = player["_createStream"];
  player["_createStream"] = function() : Void
  {
    tmp.apply(this, null);
    this._ns[nsFuncName] = func;
  };
}

With this one, only the instance of the VideoPlayer you are interested in will be patched, the function to add to the NetStream is dynamic and so is the actual function to call:

var flvp : FLVPlayback = FLVPlayback(mc.flvp);

fixVideoPlayerInst(flvp.getVideoPlayer(0), "onCaption",
Delegate.create(this, this.onCaption));

Which also allows for a metadata function per video stream (if you have multiple VideoPlayers in the FLVPlayback instance).

Which is probably a little cleaner.