Tutorial 3 – Simple Video Processing

Requirements:
Tutorial 2 – Getting Camera Information


From the previous tutorial, we left off with the ability to get information about cameras in the system.

In this tutorial we’ll actually stream video from them and do some very simple video processing.

First, add the following Reference to your .NET application from the SDK folder:

  • AxHeimdallLib.dll

Also, add the following system Reference to your .NET application:

  • System.Windows.Forms

The main component that we’ll be dealing with throughout this tutorial is AxAuga. AxAuga is the streaming/decoding/rendering part of the SDK. It actually has quite a bit of functionality, but in this tutorial, we’ll focus on its ability to stream and decode the video.

First off, we need to create an instance of AxAuga:

Because we’re using an ActiveX Control without a Windows Form, we must force AxAuga to create its own Windows Handle:

As noted in the comment, CreateControl does not need to be used if the AxAuga instance is being added as a Control to a Windows Form. In this tutorial, we don’t need a Windows Form, so we’ll omit it.

Now that we have created the Control, we need to give AxAuga the privileges to stream, like so:

SetCameraIDispatch takes the ID of the camera, and the camera’s IDispatch interface. This gives AxAuga direct access to the camera for privileges to stream.

Once we have streaming privileges, let’s create the callback class that AxAuga will callback into once a frame is received. The class will implement the IVideoHookCallback interface, which is what AxAuga requires as its callback receiving type:

IVideoHookCallback is an interface that requires implementation of two methods: CreateFrame, which returns a IVideoHookFrame, and Callback which takes a IVideoHookFrame. Let’s create a simple definition of both:

CreateFrame returns an IVideoHookFrame that will be given to Callback with a complete frame; in this case we won’t be doing anything fancy—just new the default implementation and return that. Callback is called on for every frame received by AxAuga with the object returned by CreateFrame; in this case we’ll just print out various pieces of information about the frame. IVideoHookFrame holds all of the information about the video frame:

Start: The time the frame was recorded in number of 100-nanosecond intervals since January 1, 1601 (UTC) or Window FileTime.
End: The time when the frame is no longer valid in number of 100-nanosecond intervals since January 1, 1601 (UTC) or Window FileTime.
Data: A byte array of the data in the frame. What format this buffer is in depends on the Format setting. This is considered the ‘primary’ buffer. It will always have data if the frame is valid.
Cb: A byte array of the Cb data if the format is planar.
Cr: A byte array of the Cr data if the format is planar.
Width: The width of the frame.
Height: The height of the frame.
Format: The format of the frame. Possible formats and their values:
0x1 – BGR
0x10 – YUV2
0x100 – RGBA
0x1000 – ABGR
0x10000 – YUV411
0x10000 – YUV420
0x100000 – YUV422
0x1000000 – GRAY
0x8001 – JPEG
0x8002 – H264
0x8003 – MP4
0x8004 – MXPEG
0x8005 – H263
-1 – Unknown
Codec: The original codec of the frame. Possible codec types and their values:
0x8001 – JPEG
0x8002 – H264
0x8003 – MP4
0x8004 – MXPEG
0x8005 – H263

Now that we have a definition for our callback class, let’s setup the stream and define a few command constants that we’ll use later:

AxAuga’s StartStreamingToProcessorObject sets up the stream with a callback object. It takes the URL to stream the video from, a IVideoHookCallback object, the max width, the max height, and the pixel format. It returns a stream ID, which we’ll use for controlling the stream. The max width and height tells AxAuga to restrict the frames to a specified width and height; a 0 for either tells AxAuga not to restrict for the respective dimension. The pixel format argument tells AxAuga to send the frame in a specific color space. Supported pixel formats include:
0x1 – BGR
0x10 – YUV2
0x100 – RGBA
0x1000 – ABGR
0x10000 – YUV411
0x10000 – YUV420
0x100000 – YUV422
0x1000000 – GRAY
0x8000 – Frame passthrough

The frame passthrough pixel format (0x8000) will tell AxAuga to pass the encoded frame straight through to the callback. The value for the pixel format in the callback will match one of the codecs above.

Finally, with all of the callback setup completed, now we tell AxAuga to start the stream:

AxAuga’s VideoHookPTZCommand takes the ID of the stream in which to use the command on, the command ID, and 3 arguments with the command. The command ID, START_LIVE, is used to start the live stream, and requires no arguments (hence the 3 argument parameters set to 0). After this, call the frames should be set to our Callback function in our callback object.

Lastly, we stop the stream, unregister our callback, and shutdown the AxAuga instance:

First, we issue the stop live stream command to AxAuga via VideoHookPTZCommand with the command ID STOP_LIVE. Then, we unregister the callback object using AxAuga’s StopStreamingToProcessor method, which takes the stream ID to stop. Finally, we shutdown the AxAuga instance using the method Shutdown. These steps are necessary during stream shutdown to ensure that no resources are leaked.

Screenshot of the final program:
Series-2-Tutorial-3

That concludes this tutorial. You can now setup a video frame processor with minimal stream controls.


Next:
Tutorial 4 – Playback Control