MP4 Demultiplexer Library

Table of Contents


Library Overview

This section explains the application programming interface for the MP4 Demultiplexer library and how to use the library when creating user applications.

The library takes an MPEG-4 File Format, Version 1-compliant stream as input and gets information (such as the position and timestamp from the header data) for extracting H.264 video data and MPEG-4 AAC (AAC-LC, HE-AAC v1, or HE-AAC v2) audio elementary streams.

Figure 1: Library Overview

The output of 3GPP TimedText Format Data is beyond the scope of this discussion.

Package Contents

The package content of the MP4DMX library is as follows.

Filename Description
mp4demux.a MP4 Demultiplexer library.
Header files defining values including the API function names and parameter values.

Resources Used

The library uses the following resources.

Description Required Size (Bytes)
MP4 Demultiplexer library. 256
Memory used by the library on the application side.
Each stts-Box, stsz-Box, ctts-Box. When set to read in 32-KB partitions. 630
When set to no partitions. 1600; see Note
If no-partitions is set, the memory size depends on the MP4 data file size (size of each Box). The value in the table is for reference only, calculated for a 1.56-GB MP4 file (stts Box: 35.8 KB, stsz Box: 374.3 KB, ctts Box: 658.5 KB).

Implementation Notes

To use this library, the user application must secure the memory area for the library to use. The amounts of memory used by the library are given below.

Memory That Must be Reserved by User Application

Description Size (Bytes)
Resource required.
Each stts-Box, stsz-Box, ctts-Box. When set to read in 32-KB partitions. 256 + 630
When set to no partitions. 256 + 1600; see Note
Memory required for reading the MP4 file into user application memory. 32 or more.
If no-partitions is set, see Resources Used regarding memory size.

Library Specifications

The MP4 Demultiplexer library provides functions to obtain the information needed to extract H.264 video and MPEG-4 AAC (AAC-LC, HE-AAC v1, HE-AAC v2) audio elementary streams from files conforming to MPEG-4 File Format, Version 1, according to the specifications given in the following sections.

Input Specifications

The format for input streams is given below.

Some non-typical stream formats may not be supported.

MP4 Data

Streams that comply with the following formats can be accepted as input MP4 data.

Encoding Formats

In general, this library supports streams from files conforming to the MPEG-4 File Format, Version 1 specifications given above. The library does not support MPEG-4 File Format, Version 2, but does include some support for fragment files and 64-bit compatible file formats.

If your MP4 data cannot be demultiplexed with this library or cannot be played using the sample program, contact Nintendo support and include a sample of the MP4 data.

Structure of MP4 Data

The input data structure is shown in the diagram below.

Figure 2: Structure of Input MP4 Data

Output Specifications

The MP4 Demultiplexer library outputs only offset positions from the beginning of an MP4 data file and the data size. Using this information, user applications can extract elementary stream (ES) data.

Applications can get the following ES data as output from the library.

H.264 Video Data

Applications can get H.264 video data in the form of an H.264 elementary stream. The following H.264 elementary stream specifications are supported.


Profiles and levels


Frame rate

Stream structure

File size

MPEG-4 AAC Audio Data

Applications can get audio data in the form of an MPEG-4 AAC (AAC-LC, HE-AAC v1, HE-AAC v2) elementary stream. The following MPEG-4 AAC specifications are supported.



Frame format


Sampling frequencies

Number of channels

Stream structure

Library Functions

Demultiplexing Data Flow

The following diagram shows how data flows during demultiplexing.

Figure 3: Data Flow During Demultiplexing

Demultiplexing Process

Basic operation of the library comprises two processes for demultiplexing a stream: the ordinary process, which plays an MP4 data file back from the beginning, and the special process, which uses chunk offset information to fast-forward or rewind a bitstream. The ordinary process always starts playback at the beginning of a file regardless of seek points. Special process always starts playback from a seek point.

Ordinary Process

The ordinary process starts by parsing an MP4 data file's FileTypeBox and MovieBox information to get information about the MP4 file. Then execution is performed (MP4DMXExecute), outputting information that can be used to extract ES data to the callback routines, based on the parsed information. If the data contains one or more MovieFragmentBox elements, when all of the data in the current fragment has been output, the next MovieFragmentBox must be parsed by finding (MP4DMXFindHeader) and parsing (MP4DMXReadHeader) the header. Execution (MP4DMXExecute) (or sample extraction) ends when all information regarding elementary streams within the specified range in the MP4 file has been passed to the callback routine and the callback routine has extracted the stream within the specified range.

The following diagram provides a flowchart of the ordinary process.

Figure 4: Ordinary Basic Process Flowchart

The following diagram shows the ordinary process when the MP4 data file contains one or more MovieFragmentBoxes.

Figure 5: Ordinary Process Flowchart (When Data Includes MovieFragmentBox)

The following diagram shows the process when boxes are split using a specified read size.

Figure 6: Ordinary Process Flowchart (When Data Contains Boxes Split by a Specified Read Size)

Special Process

In addition to performing the steps used in the ordinary process, for the special process, the offset of the seek point reached in current execution (get current position, MP4DMXExecute) must be obtained first. Then execution (or sample extraction) must be resumed using the offset value obtained. The following figure shows a flowchart that illustrates the special process.

Figure 7: Overview of the Special Process

Basic Library Functions

User applications using this library can perform demultiplexing operations using the seven steps described below (initialization to library closing).

  1. Initialization (MP4DMXOpen).
    Initializes the library, securing and allocating the resources that will be used. Allocation is performed according to the memory allocator obtained by the user application.
  2. Begin Multiplexing (MP4DMXBegin).
    Initializes library variables and performs other internal initialization.
  3. Get Header Position (MP4DMXFindHeader).
    Retrieves the positions (offset from the beginning of the MP4 data file) and sizes of FileTypeBox, MovieFragmentBox, MovieFragmentRandomAccessBox, and MovieFragmentRandomAccessOffsetBox records within the MP4 data file.
  4. Parse Header (MP4DMXReadHeader).
    Parses headers for FileTypeBox, MovieBox, MovieFragmentBox, MovieFragmentRandomAccessBox, and MovieFragmentRandomAccessOffsetBox records.
  5. Perform Demultiplexing (MP4DMXExecute).
    Performs seek and demultiplexing within the MP4 data file.
  6. Complete Demultiplexing (MP4DMXEnd).
    Outputs any data not yet output.
  7. Close Library (MP4DMXClose).
    Releases any resources secured during initialization.

Utility Functions

The main purpose of the basic API is to handle the Box structures located directly under the root directory of an MP4 data file. The following utility functions are also provided for retrieving information and parsing lower-level Box structures.

See Also

MP4 Demultiplexer Sample Demos

Revision History

2014/07/30 Initial version.