AX Programming Model

Introduction

The AX audio subsystem operates under strict real-time requirements. Each frame of processing must complete within its allotted time or else the user may experience unpleasant "click" or "pop" artifacts due to discontinuities in playback. To that end, AX does most of its processing within the context of an AX Driver thread that runs at a higher priority than normal application threads. One time every audio frame (3ms), this thread is woken up by a hardware interrupt.

The audio system is double-buffered. While the hardware plays the audio from the previous rendering frame, the audio driver and application code will render audio for the next frame.

As of this writing:

In the future, AX will be multicore safe and support core affinity.

Playing a Sound

The basic procedure for playing a sound is as follows:

  1. Load the sound data from storage into main memory.
  2. Acquire a voice from AX.
  3. Configure the parameters for the voice. At a minimum:
  4. Free the voice after playback has completed.

For an example of this, see the axsimple demo.

The AX Interrupt

In order for audio playback to occur without glitches, AX must maintain a steady flow of output. To do this, it has an interrupt that occurs once per audio frame (3ms). When this interrupt occurs, the AX driver thread is woken up to do the necessary processing for the next frame of audio.

The timing of the default pipeline process is illustrated below:

Starting from Sound2, a four-stage pipeline process is added to allocate more computation on DSP and reduce the load on PPC. The timing is illustrated below:

User can choose which pipeline process to run by specifying the parameter in AXInitWithParams.

The AX driver thread performs the following steps in both processes:

Sync Voice Parameters

If any changes were made to voice settings, they are synchronized with renderers for AX. The voices will be rendered with these synchronized settings.

Changes to protected voices will not be synchronized until the protection is lifted. For more information, see the Synchronization section.

Voice Generation

After voice parameters have been synchronized, AX begins the voice generation step of audio rendering. AX commands the DSP renderer to begin the work assigned to it, then runs the PPC renderer.

AUX Callbacks

After PPC voice generation is complete, AX calls any AUX callbacks set by the application, allowing special effects to be applied to the audio of AUX buses. The callback provides a sample buffer that contains of the results of voice generation from the previous frame. At this point, the DSP may still be doing the current frame's voice generation. To apply special effects, modify the provided sample buffer. The contents of the buffer at the end of the callback will be mixed into the final output for the frame.

Frame Callbacks

Any frame callbacks that have been registered with AX are called at this point. This is an ideal place to perform any processing that must occur once per audio frame.

Bus Mixing

The AUX buses and main bus are combined at this step, as described in the Rendering Model's mixing section. Each channel of each voice is left with one output buffer apiece.

Output Formatting

Each of the mixed output buffers is formatted for its final output. This process is described in detail in the Rendering Model's output formatting section.

AX Callbacks

Frame Callbacks

Frame callbacks occur once every audio frame in the context of the AX driver thread.

Most AX Function calls may be made either at the Application level Threads or from within the Frame Callback function. It is recommended that one or the other method is chosen and used consistently. Criteria for choosing which method to employ are:

AUX Callbacks

Applications may set one AUX callback for each AUX bus of each device. After voice generation has completed and audio has been mixed to each bus, AX calls the AX callbacks. Each callback receives the sample buffers for its corresponding AUX bus and has free reign to modify them. After the AUX callback returns, the contents of the bus are mixed with the output from the device's other buses to form the final mixed audio data.

Final Mix Callback

The final mix callback is called after all of the AUX and main buses have been mixed together. This callback receives pointers to the final mix. Applications may both read and write to these buffers. After this callback, the resulting buffers are formatted for output and then sent to the audio hardware.

Voice Drop Callback

When a voice is acquired, it is possible to provide a voice callback. If the voice is forcibly dropped by AX, this callback is invoked. After the callback returns, AX will call AXFreeVoice to complete the drop. The Application must ensure no further references to a dropped voice can be made after the callback. A voice may be dropped for the following reasons:

Exceed Callback

The exceed callback exists primarily for debugging purposes. It is only called if the "Mix Main + Auxes" step of processing is not reached before the end of the 3ms audio frame. If this happens, it is likely that audio glitches will occur.

Synchronization

For voice-related data structures, AX provides lightweight synchronization mechanisms to protect data access between application threads and the AX user callbacks:

AX Protection

It is recommended that applications make all AX calls within the context of an AX callback. However, in some cases, applications may prefer to make calls outside the context of these callbacks. For these cases, AX provides a lightweight synchronization option: voice protection. As long as a voice is protected, changes to the settings of the voice will not be committed to the renderer. By protecting a voice, applications can guarantee that a set of updates to a voice are performed atomically, preventing problems due to voice generation interrupting a set of updates.

It is possible to nest instances of voice protection. If protection for a voice is asserted multiple times, its protection must be lifted an equal number of times before it is unprotected.

While in the context of an AX callback, it is not possible or necessary to begin protection for a voice. Callbacks are executed from the AX driver thread's context, so there is no possibility of interruption.

User Protection

AXUserBegin and AXUserEnd enable a user protection state. While in a user protection state, any voice that has any parameters modified will automatically be protected for the duration of the user protection state.

To determine whether an AX user protection state has been asserted, call AXUserIsProtected.

Single Voice Protection

Use AXVoiceBegin and AXVoiceEnd to protect a single voice.

It is possible to check whether a voice is protected by using AXVoiceIsProtected.

Best Practices

Queue/Commit

User threads may be interrupted at any time by the AX driver thread. To avoid synchronization issues, it is highly recommended to make most AX function calls within the AX frame callback, which operates within the interrupt context. This prevents the need to make use of AX's synchronization primitives. One model way of making all AX calls occur within the AX callback is queuing and committing. The concept for this is:

Callback Responsiveness

It is important to have all AX callbacks return in a timely manner, as they run in the AX driver thread's context. The AX driver thread cannot continue its work while a callback has not returned. Additionally, no blocking API's may be called from within a callback. Doing so is prohibited by the standard OS practices, and for AX, may result in priority inversion problems as well as broken audio output.

Programming Notes

Wii Remotes

AX supports the generation of audio samples for Wii Remotes. However, unlike the TV and DRC, AX does not handle the output of this sample data to the Wii Remotes. The process for using AX to play audio to a Wii Remote is as follows:

For an example of this, see the wpad_spdemo sample.

Additionally, remotes have an optional extra filter stage. Audio mixed for Wii Remotes will have this filter applied if it is configured. This filter can be selected and configured with the following functions: AXSetVoiceRmtIIR, AXSetVoiceRmtIIRCoefs

See Also

General Topics

Basic Thread Scheduling
Cafe Core OS (COS) Overview
COS Shell Commands

AX Functions

AXInit
AXAcquireVoice
AXFreeVoice
AXSetVoiceType
AXSetVoiceOffsets
AXSetVoiceSrcRatio
AXSetVoiceSrcType
AXSetVoiceVe
AXSetVoiceDeviceMix
AXSetDeviceVolume
AXSetAuxReturnVolume
AXSetVoiceState

AX Synchronization

AXVoiceBegin
AXVoiceEnd
AXVoiceIsProtected
AXUserBegin
AXUserEnd
AXUserIsProtected

AX Applications and Tool Chain

Sound Pipeline Tool Chain
AX Applications Overview
Sequencer Overview (SEQ)
Synthesizer Overview (SYN)
Articulator Overview (AXART)
Mixer Overview (MIX)
Aux Effects Overview (AXFX)

Revision History

2015/06/01 Fixed links to Basic Thread topic.
2013/10/30 Copy-edits.
2013/05/08 Automated cleanup pass.
2012/08/01 Cleanup Pass.
2011/02/21 Initial version.


CONFIDENTIAL