Combining Anti-Aliased and Single-Sampled Rendering

There may be times when it is advantageous to combine multi-sampled anti-aliased (MSAA, or just AA) and single-sampled (1x) rendering in the same image. For example, rendering a pixel-accurate UI in front of anti-aliased geometry from the game world. GX2, the Wii U graphics API, does not allow for mixed-mode sampling of a single surface. That is, you may not treat the same surface as if it were both AA and 1x. However, you may convert an AA-buffer into a 1x-buffer. This topic describes several techniques for combining multi-sampled and single-sampled rendering on the Wii U. Code for these examples may be found in aainto1x.cpp, located in the $CAFE_ROOT/system/src/demo/gx2/misc/resolveAA directory. The example performs each of the resolve methods depending on the DrawMethod enum specified by sceneData.technique, which may be changed by pressing left and right on a debug controller.

Separate Color Buffers

Figure 1: Separate AA and 1x color buffers blended together.

The most straight-forward method for rendering 1x and AA in the same image is to keep separate color buffers for each type of rendering. These color buffers may then be blended in a shader. This may be sufficient for some cases. Because the 1x color and depth buffers are distinct from the AA color and depth buffers, information from one set may not be used when rendering the other. However, the information from the AA buffers may be reused when performing the 1x rendering by resolving the AA buffers.

Reuse the Color Buffer via GX2ResolveAAColorBuffer

Figure 2: The AA color buffer is converted to a 1x color buffer by GX2AAResolveColorBuffer.

To resolve is the process of converting an AA color buffer into a 1x color buffer. The prerequisites for taking advantage of this are:

The code must perform all of the AA draw calls before the 1x draw calls. Interleaving calls do not work as intended after the source color buffer is resolved because any operations on it do not affect the destination color buffer.
The depth buffer from the AA rendering may not be reused by the 1x rendering – only color buffers may be resolved by GX2.
The GX2 resolve operation supports only certain surface formats – see the definition of the function GX2IsResolveSupported.

In the example, the DrawScene function performs the complete operation. It sets up an initial AA color buffer and depth buffer (SetColorDepthBuffers), renders the AA geometry (DrawGeometry), resolves the AA color buffer and sets up the resulting 1x color buffer and a fresh 1x depth buffer as render targets (Resolve), and then renders the next set of geometry single-sampled (DrawGeometry).

The Resolve function resolves the color buffer by converting an AA color buffer into a 1x color buffer. Until this point, the rendering was done on the AA color and depth buffers. This function sets the resultant 1x color buffer and depth buffer as the new render targets. Since the resultant color buffer now contains the down-sampled version of the AA color buffer, GX2ClearColor should not be called on it – doing so wipes out that information.

After the call to GX2ResolveAAColorBuffer, the AA color buffer is no longer needed for this frame, and any changes made to it do not affect the 1x copy. At this point, it could be freed and reused for another purpose. Additionally, GX2ResolveAAColorBuffer can act in-place. For example, the source and destination color buffers share the same surface.imagePtr). In the example, resolving a 720p 4x MSAA 32-bit color buffer in MEM1 takes 420 microseconds as measured by the GX2Sample*GPUCycle functions.

Reusing the Color Buffer via Shader

Figure 3: The AA color buffer, treated as texture, is resolved to a 1x color buffer by a shader.

A shader may also perform a resolve. To resolve by using a shader, a texture is created from the source color buffer data. The texture is sampled by the shader in a full-screen pass, and the values are down-sampled and written out to the destination 1x color buffer. The prerequisites for taking advantage of this are:

The code must perform all of the AA draw calls before the 1x draw calls. Interleaving calls do not work as intended because, after the source color buffer is resolved, any operations on it do not affect the destination color buffer.
The destination 1x color buffer must be allocated separately from the source AA color buffer. Resolving via shader cannot be done in-place.

Resolving via shader is the only way to resolve a color buffer with a surface format that is not supported by GX2ResolveAAColorBuffer, and the only way to convert surfaces that are not color buffers.

The function InitQuadBuffers creates a GX2Texture from the source AA color buffer (TextureFromAASurface). The texture is sampled by the pixel shader using a sampler2DMS. InitQuadGeometry configures the triangles necessary for rendering a full-screen pass, with texCoords to be used in the pixel shader.

As in the previous method, Resolve sets the render targets to the 1x color buffer and depth buffer. This method calls ShaderResolve, which performs the following.

Expands the AA Color Buffer (GX2ExpandAAColorBuffer), filling in the AA color buffer with the information that is cached in the Aux buffer.
Draws a full-screen quad to invoke the pixel shader on the entire screen. To perform this, it samples each sub-pixel of the MSAA color buffer from step 1 with the GLSL function, texelFetch, averages the results, and then writes them to the new buffer.

In the example, resolving a 720p 4x MSAA 32-bit color buffer in MEM1 takes about 2800 microseconds. The two primary contributors to this time are expanding the color buffer and sampling the texture in the shader.

Reusing the Depth Buffer via Shader

Figure 4: The AA color buffer and depth buffer, treated as textures, are resolved by a shader.

The method of reusing the depth buffer via shader is an extension of the previous method. This technique allows the depth information from an AA-rendered scene to be reused when rendering with single-sampling afterward. In the example, the SS circles are farther from the camera than the AA striped triangles. When the depth information is not preserved, the circles, which are drawn last, appear to be in front. When depth information is preserved, the triangles appear to pass in front of the circles.

In addition to the setup in the previous section, a second GX2Sampler and GX2Texture are set up to sample the AA depth buffer. The texture for the depth buffer is created in the same way, and the compSel field is setup in GX2_COMP_SEL_XXXX since the texture uses the single-channel format, GX2_SURFACE_FORMAT_R16.

Since the depth buffers use a tiling format that is different from color buffers they must be converted first to be readable as a texture via GX2ConvertDepthBufferToTextureSurface. The shader then samples from both the AA color and depth textures, writing to the new single-sampled color and depth buffers.

This process is time-intensive, weighing in at around 5700 microseconds to resolve the 720p 4xMSAA 32-bit color buffer and 16-bit depth buffer, both in MEM1. If GX2ResolveAAColorBuffer supports the color format, you may use it on the color buffer, and then disable color writes and resolve with a shader that only samples and writes depth values. Given that the time to write to the color buffer and the depth buffer is nearly double that of just the color buffer, we may expect the depth buffer resolve to take about as long as the color buffer resolve (2900 µs).

Depth Buffer Down-Sampling Method

The example code down-samples the depth buffer by averaging the subsamples. This proves about 100 microseconds quicker than taking the maximum or minimum values. Due to the sparse scene, all methods are performed relatively equivalently in terms of quality.

Appendix

The multi-sample anti-aliasing and single-sample rendering example demonstrates resolving AA color and depth buffers to single-sampled surfaces.

Running the demo:

In cafe.bat:
1. Change the CAFE_ROOT variable to point to the correct directory for the SDK with the Wii U CPU Profiler installed.
2. If Cygwin is not installed at C:\cygwin, change the CYGWIN_PATH variable in cafe.bat to point to the correct directory.
Ensure that the SDK and CAT-DEV are already configured according to the SDK guides.
Double-click cafe.bat.
At the command prompt, type cd $CAFE_ROOT/system/src/demo/gx2/misc/resolveAA.
At the command prompt, type make run.
The demo runs and outputs the scene to the TV:
1. Press the left and right buttons on the +Control Pad on a Debug Controller to switch rendering techniques.
2. Press the B Button to stop the program.
After stopping the program, at the command prompt, type cafestop to stop the CAT-DEV.

When changing rendering methods, note the presence of sampling artifacts in the gray lines (smooth lines indicate MSAA rendering), and the position of the striped triangles with respect to the circles (triangles in front indicates depth information was used for rendering the circles).

Revision History

2014/03/27 Fixed paths to demo and removed 'this paper'.
2013/08/05 Converted from PDF to HTML format.

CONFIDENTIAL