Network Overview

Table of Contents

Introduction

The Cafe SDK provides TCP/IP networking via the socket interface, similar to the popular BSD network programming API. The Cafe network stack is organized in two parts:

Getting Started

Before starting on a new CAT-DEV system, network configuration is performed using System Config Tool and saved in NAND memory for the future use. This is equivalent of performing network configuration of Windows, Linux or Mac OS X based systems using their respective UI.

It is mandatory for network programs to first using AC (Auto Connect) library before using ANY other networking library. AC library ensures that network connectivity is up and running so that other networking libraries can send and receive network traffic. Refer to the AC section for descriptions of the AC library APIs.

After this point, programs can use network libraries of their choice including "socket API", "libcURL", "nSSL" or any other networking middleware like NEX etc.

Socket Programming

The Cafe network stack offers rich networking features including (but not restricted to):

Refer to the socket demos for examples.

Notes On Porting Network Code From POSIX Compliant Operating Systems

Socket API

Socket library requires initialization (e.g., SOInit call is used to initialize socket API, SOFinish call is used after an application is finished using socket API). There calls should be made BEFORE any socket API is used. It is recommended that these calls be made only once at the beginning of the program and when the program is being terminated.

Socket API in Wii U stack is designed to match closely with the BSD socket API. Socket API is thread-safe. It provides both blocking and non-blocking IO semantic. "select" call provides UNIX like mechanism to wait on multiple socket descriptors. Refer to individual man-pages for the error codes returned by socket APIs.

errno

Wii U SDK as a whole does not use UNIX style global "errno" to return system call errors. However, Socket API errors are returned in a global errno variable that is thread-safe. The application can continue to examine this variable after a socket call returns failures. Application developers should use the error numbers defined in net_types.h which use SO_ prefix. Network stack does not return error numbers found in ghs/ansi/errno.h. It is part of the porting effort, application developers should map the error number macros used in their code to those found in net_types.h. While socket APIs will return most commonly used errors (see man-pages), the list of errors may not be as exhaustive as other platforms. Other platforms support different OS features like SIGNAL, PIPEs etc which have error number representation on other system. These will not apply to Wii U network errors.

Network Link Loss

There is a significant difference in which how network stack reacts to the loss of network connectivity. On detecting a network cable being unplugged or on the loss of link to a wireless access point, network stack will terminate all sockets belonging to the application. Threads which were blocking in socket API will be woken up with an error SO_EABORTED. Subsequent calls on the terminated sockets will return SO_ENOTSOCK error. The application is expected to allocate the socket again and resume operations. It is not required to reinitialize the socket library using SOinit.

Multithread-safe Socket API

The Socket API is multithread-safe. In addition to that, multiple threads can operate on the same socket (either in blocking or non-blocking mode) - for example one thread sending on a socket and another thread receiving on the same socket.

If a socket is closed from one thread while there are blocking operations on the same socket from other threads, the blocked operations are aborted.

If multiple threads make a send call on the same socket, the usual send semantics still apply; i.e., the send call may return with fewer bytes sent than the requested amount and send must be called again to send the remaining bytes. This also means that if multiple threads send on the same socket at the same time, the data from these threads may get interleaved as they are sent.

Performance Tips

Buffer Alignment

The application can improve performance (by avoiding internal buffer copying) by using buffers which are aligned with the PPC cache line size (defined by PPC_IO_BUFFER_ALIGN) for data transfer operations (e.g., SOSend, SOSendTo, SORecv, SORecvFrom). Both buffer address and size must be aligned. For an implementation example, see the UDP echo demo (system/src/demo/so/ac/udpecho.c).

Socket Buffer Sizes

The default TCP and UDP socket buffer sizes are 8 KB for the send buffer and 8 KB for the receive buffer. If the application desires, it can increase the socket buffer for specific sockets using the SOSetSockOpt function.

The send and receive buffer for TCP sockets can be increased up to 128 KB if the socket has SO_WINSCALE option set, otherwise the maximum limit is 65535 bytes.

For UDP receive buffer the maximum allowed size is 65535 bytes. There is no affect on increasing UDP socket send buffer from the default size because after the UDP packet is transmitted on the link, it is no longer buffered.

NOTE:
For optimal system performance, it is recommended that socket buffer sizes be chosen judiciously. The value should be chosen based on the application's expected buffering requirement for a connection rather than just setting it to the maximum possible value.

For more information on setting socket options, refer to SOSetSockOpt.

Guidelines

  1. Efficient Memory Utilization

    The network stack buffer pool is mainly consumed by the application's send and receive socket buffers. On the receive path this means when network data is received, the network stack buffer pool memory remains tied up in the socket receive buffer till the application reads the data from the socket receive buffer into its own memory. The application is required to read data actively whenever they expect to receive data otherwise they can negatively affect other network clients and system network performance.

    Instruction

    1. To Avoid Exhaustion

      Using SOSelect function is a good way to optimally receive data from the network. In general, following behavior of the application is NOT encouraged and should be avoided.

      • Unnecessarily setting big receive socket buffer sizes via SOSetSockOpt function. Instead, socket buffer sizes should be configured properly based on the application's real requirements.
      • Reading data in a low priority thread. Instead, thread priority of network data transfer should be configured properly to get it to run sufficiently fast to keep up with incoming data.
    2. To Check Consumption

      The application can obtain socket buffer consumption with SOGeLibOpt function. It returns one of the consumption levels (amount of consumed packet buffers by the application) listed below and the application is required to take an appropriate action depends on the obtained level.

      • SO_SOCKBUF_CONSUMPTTION_LOW
        The application can continue with what it is doing and it can potentially increase its usage if required (e.g. creating new sockets etc.).
      • SO_SOCKBUF_CONSUMPTTION_MED
        The application can continue with what it is doing, but it should avoid increasing its usage.
      • SO_SOCKBUF_CONSUMPTTION_HIGH
        The application should try and read data from socket buffer to its own buffer, and it should not increase its usage.
      • SO_SOCKBUF_CONSUMPTTION_MAX
        The application has to read data from socket buffer to its own buffer urgently till the usage level is indicated to be MED or LOW.

      [NOTE]
      It is not recommended to call SOGeLibOpt function persistently at short intervals. It has to be used to check whether the application is unnecessarily consuming/holding socket buffers at some point. We recommend to put at least 15 seconds interval between each function call.

      Refer to $CAFE_ROOT/system/src/demo/so/ac/sockbuf_check.cpp and Networks Stack Demos overview for further API usage.

    Background

    The network stack operates with a fixed amount of memory. It initializes a buffer pool from this memory during system initialization and this buffer pool is shared by all network clients including network stack itself for its internal use. The network stack obtains free buffers from this buffer pool when network clients send and receive data, and frees the buffer is not needed anymore.

    Here are some examples when memory is consumed from the buffer pool:

    • Consumed when reassembling fragmented IP packets.
    • Consumed when UDP packet is sent out by the application, freed when the packet is sent out from the network interface.
    • Consumed when TCP packet is sent out by the application, freed when the network stack receives TCP acknowledgment to this packet.
    • Consumed when the network stack receives TCP packet to store it in a socket receive buffer, freed when the application reads it from the socket receive buffer.

    Important Behaviors

    Here are some important behaviors related to the buffer pool in the network stack.

    • Setting socket buffer size does not pre-allocate or pre-reserve memory for the socket buffer. The size places a limit on how much data can accumulate in the buffer. For example, when the socket receive buffer is set to 64KB, no memory is allocated at that time. This just means that the socket is allowed to consume at maximum 64KB of memory for its receive buffer. The memory is consumed as packets arrive for this socket, one packet at a time up to a maximum of 64KB. When the application reads data from the socket receive buffer, the memory consumed by the socket buffer reduces, reaching zero when all data in the receive buffer has been read.
    • TCP receive window size is calculated based on the socket receive buffer size (not based on the available memory in the network stack). For example, if a receive buffer size of the specific socket is set to 64KB and no data is received yet, 64KB of window size will be advertised for the connection related to this socket. But if a situation arises where another socket of this application or other applications consume all the free buffers in the network stack, then advertised window size of this socket will be temporarily set to 0 because the network stack is not in a position to receive any more packets. After free buffers are available again as applications read data from socket buffers, the socket connection's advertised window size will be increased.
    • For efficiency, the network memory buffer pool uses fixed buffer sizes of various ranges. Depending on the size of packets, a different sized memory buffer is used. In some cases the memory buffer used may be bigger than the network data packet. The actual memory consumed in the network stack may be bigger than the size of data received by the application (for example as obtained by querying amount of data in the socket receive buffer).

  2. Guidelines For All HTTP Network Applications

    We recommend the application which does HTTP data transfer to use Libcurl in Cafe SDK because it handles HTTP packets with stored network setting in the system automatically.

    It is allowed to perform HTTP data transfer in the application's own way (e.g. using Libcurl which is not provided by Cafe SDK). But such the application is required to apply proxy setting which is stored in the NAND before starting data transfer. The system stores following proxy information in the NAND.

    • The flag which indicates whether use proxy or not
    • Proxy server address
    • No proxy addresses
    • Proxy port number
    • Proxy authentication method
    • User name for the proxy authentication
    • Password for the proxy authentication

    These information have to be applied before starting HTTP data transfer because Wii U users can configure proxy setting in Wii U menu and expects it gets applied for all the HTTP data transfers.

    Instruction

    The application can obtain proxy setting with nn::Result nn::ac::GetStartupId and nn::Result nn::ac::ReadConfig APIs. Here are the steps for obtaining proxy setting.

    • Read startup profile ID by nn::Result nn::ac::GetStartupId API
    • Read network configuration by nn::Result nn::ac::ReadConfig API
      • The ID obtained by nn::ac::GetStartupId must be specified to the argument of this function
    • Call nn::ac::GetProxyUse with obtained network configuration above to see if proxy is enabled
      • If nn::ac::GetProxyUse returns AC_PROXY_STATE_DISABLE, then the application does not have to do anything because proxy is disabled
      • If nn::ac::GetProxyUse returns AC_PROXY_STATE_ENABLE, then the application needs to apply each proxy setting described below
        • Call nn::ac::GetProxyHostName with obtained network configuration above
          • Proxy server address can be obtained with this API
        • Call nn::ac::GetProxyNoproxyHosts with obtained network configuration above
          • No proxy addresses can be obtained with this API
        • Call nn::ac::GetProxyPort with obtained network configuration above
          • Proxy port number can be obtained with this API
        • Call nn::ac::GetProxyAuthType with obtained network configuration above
          • Proxy authentication method can be obtained with this API, if it is AC_PROXY_AUTH_TYPE_NONE then no need to apply user name and password described below
        • Call nn::ac::GetProxyUserName with obtained network configuration above
          • User name for the proxy authentication can be obtained with this API
        • Call nn::ac::GetProxyPassword with obtained network configuration above
          • Password for the proxy authentication can be obtained with this API

    These functions are allowed to use only for development purpose. But such the application described above is allowed to use it even on the product application. Refer to AC for further information.

Revision History

2014/02/06 Added "Efficient Memory Utilization" section in Guidelines. "Memory Utilization in the Network Stack" was merged into there.
2013/09/25 Added "Guidelines" section.
2013/08/16 Added "Memory utilization in the network stack" section.
2013/05/16 Added note about Socket Buffer Sizes in Performance Tips section.
2013/05/08 Automated cleanup pass.
2013/03/06 Improved entire description.
2013/01/23 Updates for System Config Tool
2012/07/25 Cleanup pass.
2011/10/14 Fixed for USB-Ethernet wired interface support.
2011/06/07 Modified to use AC library instead of NC library.
2011/02/21 Initial version.


CONFIDENTIAL