H.264 Decoder Library

Table of Contents

Library Overview

This section describes the contents of the H.264/MPEG-4 AVC Decoder library and its application programming interface.

The H.264 Decoder library provides a programming interface for decoding H.264 video data. The library can accept a bitstream that complies with the H.264/MPEG-4 AVC video format and output decoded image data.


Figure 1: Library Overview

Package Contents

The package content of the HE-AAC library is as follows.

Path and Filename Description
H264Dec\Release_lib_h264dec\ The H.264/MPEG-4 AVC Decoder library itself.
h264.h The header file that defines parameters, error values, function names, and other values.

Resource Usage

This library uses the following resources.

Description Required Size (KB)
User application memory used by this library. See the section on Memory.

Precautions

User applications need to allocate memory for the library to use. For information and restrictions on memory allocation, see the section on Memory.

Library Specifications

This library provides features for decoding data from H.264/MPEG-4 AVC Video Elementary Streams and has the following specifications.

Input Specifications

Input streams have the following formats and image sizes. Non-standard streams may not be supported.

Format

Input data has the following format.

Valid Image Sizes

Input images have the following sizes.

However, real-time performance may not be expected when using a stream or camera images that exceed 1080p, 24fps.

Stream Size

The maximum stream size per frame is limited as follows.

Output Specifications

Format

The following output format is supported.

Operating Specifications

Memory

User applications need to allocate the memory that the library will use.

The library uses three types of memory: work memory, frame memory, and memory for the input bitstream.

Maximum Amount of Memory Used by Library for 1920x1080 Images at Level 4.1

Type of Memory Required Size (bytes)
Work memory 40,895,487
Frame memory 3,342,336 * n
Bitstream memory 419,4304
NOTE:
These are the maximum values corresponding to Input Specifications. They vary depending on the profile identifier, level, and image size.

The size of work memory and frame memory required depend on the profile identifier, level, image size, and other usage conditions. Use a utility function H264DECMemoryRequirement to get the required memory size.

You need enough frame memory for at least six frames: one frame is used for decoding and five frames are used for the library's internal buffers. A ring buffer is normally created when the decoder outputs synchronized audio and video. Increase the amount of frame memory as needed by the user application.

If you use output from the MP4 demultiplexer or a buffer that was loaded from a file, you do not need to provide any bitstream memory in advance.

The following restrictions apply to the memory used by the library.

  1. The library must be able to write to work memory and frame memory.
  2. Work memory and frame memory must be valid from the point the library is initialized (using the H264DECInitParam function) until it is shut down (using the H264DECClose function). Bitstream memory must be valid from the point that stream input starts (using the H264DECSetBitstream function) until decoding ends (using the H264DECExecute function).
  3. Applications must not overwrite work memory and (the region being decoded) bitstream memory while the library is running.
  4. Memory regions applied to work memory and frame memory must be configured. Specifically, memory regions spanning 256 MB boundaries in physical memory cannot be applied to work memory or frame memory. Whether a region of memory can be used for work memory or frame memory can be determined using a utility function H264DECCheckMemSegmentation.
  5. The frame memory must start on a 256-byte boundary. Also, the width of the frame memory must be 256-byte aligned.

Instructions to Start Decoding

This library decodes data one frame at a time.

Because the MP4 demultiplexer outputs one frame at a time, your application can send that output directly to this library and decode it.

To decode a Video Elementary Stream, use the utility function H264DECCheckDecunitLength from this library to get the data size per-frame.

Getting Decoding Results

The configured callback function gets decoding results from the library.

The library output (the H264DECOutput data structure) passed to the callback function, including the output decoded data and VUI (voice-user interface) output within it, is valid until the callback function completes. To reference this data outside of the callback function, it must be duplicated explicitly within the callback function. However, the location indicated by Result within the decoded data output (the actual decoded output image data) is maintained after the callback function completes, so only the pointer in Result needs to be duplicated. This library does not overwrite the location indicated by Result after invoking the callback function.

To access the memory used in the callback function with this library, it is possible to set user memory as a parameter. Set an address in user memory when setting the library parameters H264DECSetParam from the user application. Parameter settings are in Parameter Settings and the H264DECOutput structure is described Library Output.

Callback functions are invoked when the user application invokes the library (H264DECExecute function) or when a stream is terminated H264DECEnd function. However, the library execution process H264DECExecute function invokes the callback function when the decoding results are stored in the frame memory. For example, when decoding results cannot be acquired normally and stored in the frame memory due to an error stream or other reasons, the callback function is not invoked.

The library invokes a callback function whenever output images are generated to output the decoded frame(s). However, exactly when the callback function is invoked is determined by the mode, which is either sequential output mode or buffering mode (as configured by the library's parameter settings with the H264DECSetParam function).

In sequential output mode, the callback function is invoked and a decoded frame is output as soon as a single image (frame) has finished decoding. Use this mode when frames are not reordered during decoding, such as with a normal Baseline Profile stream, to decode without delays and use only a small memory footprint. If a stream's frames have been reordered, the frames are output as reordered, so the user application must restore them to their original chronological order in a separate step. Because the frames do not accumulate in the library with this mode, the instruction process to terminate the stream (H264DECEnd function) does not invoke the callback function.

In buffering mode, because the frame is output after rearranging into the original video frames order, a prescribed number of frames (five frames for this library) accumulate in the library. The callback function is invoked with this mode while invoking the library (H264DECExecute function) when the decoding results stored in frame memory are accumulated in the library, and that number exceeds the prescribed number of frames. That is, when decoding results for a single frame are obtained normally, the decoding results are accumulated, and when the accumulated decoding results exceed the prescribed number of frames, the callback function is invoked, and the single frame is output. Using this mode with a stream whose frames have been reordered, a delay is introduced, but the decoding results are retrieved in the correct order.

With this mode, when exiting the library (when the H264DECEnd function is called) to output any buffered decoding result data, the library may call the callback function a single time to output the decoding results for the number of the accumulated frames.

Overview of H.264 Decoder Library

This section describes the decoding features of the H.264 decoder library.

Data Flow While Decoding

The following is a data flow diagram for decoding with the H.264 decoder library.

This library uses a hardware decoder. This reduces the amount of work that the CPU needs to do, but also means that the library waits for the hardware to finish processing. After a library function is called by an application, it will not return for approximately as long as it takes the hardware decoder to process data.


Figure 2: Decoding Data Flow

The following table provides more information on the data that is passed between modules.

Data Flows Between Modules

Module Number Data
1 Decoded H.264/MPEG-4 AVC Video Elementary Stream
2 H.264/MPEG-4 AVC frame data
3 Raw (NV12) video data

Decoding Control Flow

The following figure shows the basic control flow for decoding with the H.264 decoder library. As shown in the diagram, callback functions are invoked to get decoded results when the library is processing (the H264DECExecute function call) or a stream is terminated (the H264DECEnd function call).


Figure 3: Decoding Control Flow

Changing Data to Be Decoded

If you are decoding video data for normal playback, you repeat the bitstream set operation H264DECSetBitstream function and the execute instruction H264DECExecute on a continuous stream. If you want to start decoding from a different position in the same stream (including recovery and playback from an error stream), or if you want to decode a different stream, you need to take the following steps.

  1. To start decoding from a different position in the same stream (including returning from the error stream).
    After execution of the stream (Execute H264DECExecute) completes, terminate the stream (call the H264DECEnd function) and then restart it (call the Start Decoding H264DECBegin function).
  2. If you are decoding one stream and want to decode a different stream.
    After execution of the current stream (Execute H264DECExecute) completes, terminate it (call the H264DECEnd function), finalize the library (call the H264DECClose function), and finally reopen the library (call the H264DECOpen function) to resume.

Basic API

By following the listed nine steps, user applications can use the H.264 decoder library to decode streams.

  1. Initialize the Library H264DECInitParam - Set the work memory and initialize the library.
  2. Set the Library Parameters H264DECSetParam - Set callback function pointers and other parameters.
  3. Open the Library H264DECOpen - Initialize the library's internal state according to parameter settings. (To decode a different stream, resume by calling the H264DECOpen function after terminating the current stream (call H264DECEnd) and closing the library (call H264DECClose).
  4. Start Decoding H264DECBegin - Send an instruction to start decoding. (To resume decoding from a different position in the same stream, start the library (call the H264DECBegin function) after terminating the current stream (call the H264DECEnd function).
  5. Set a Bitstream H264DECSetBitstream - Set the bitstream to decode.
  6. Execute H264DECExecute - Decode the bitstream that you have set.
  7. Stop the Stream H264DECEnd - Notify the library to end decoding (free up hardware resources, flush and perform other processes on frames maintained internally).
  8. Flush the Stream H264DECFlush - Notify the library to flush the frame it maintains internally.
  9. Finalize the Library H264DECClose - Finalize the library.

Utility Functions

The library provides the following utility functions to make memory usage more efficient and to decode without an MP4 demultiplexer.

  1. Get Memory Sizes H264DECMemoryRequirement - Calculate the size of work memory and frame memory based on the profile identifier, level, and image size.
  2. Get Image Sizes H264DECGetImageSize - Get the size of images from a Video Elementary Stream to decode.
  3. Find a Location to Start Decoding H264DECFindDecstartpoint - Find a location from which to start decoding in the input data (the starting position of the sequence parameter set, or SPS1 - contains information related to the overall encoding of the profile, level, and stream). To decode a bitstream, first call this utility function and then start decoding from the position it returns.
  4. Find a Location to Resume Decoding H264DECFindIdrpoint - Find a location to resume decoding (the starting position of an IDR frame2 - a frame that allows random access. This frame allows you to properly decode all frames after it even if you never used information for any frames before it) in the input data. This allows you to get a position at which to start playback in the middle of a bitstream (random access).
  5. Get the Units in Which Data Is Processed H264DECCheckDecunitLength - Get the amount of data used as the base unit for decoding.


    Figure 4: Sample Bitstream Structure (With SPS Only at the Beginning)

  6. Determine Memory Application for the Library H264DECCheckMemSegmentation - Determines whether the given memory area is suitable for application as working memory or frame memory.
  7. Determine Decoding that can be Skipped H264DECCheckSkipableFrame - Determines whether the subsequent decoding is performed correctly even when the target frame is not decoded.

Image Format

NV12

The following figure shows how data is arranged when the output image format is NV12.


Figure 5: NV12 Output Image Format

See Also

ITU-T Recommendation H.264 (2012/01) Advanced video coding for generic audiovisual services
H.264 Sample Demos

Revision History

2014/07/16 Initial version.


CONFIDENTIAL