TCP/IP communication involves a complex set of protocols that make configuring a TCP/IP stack challenging for new and experienced users. This is the first of a two-part series which will break down how to configure Micrium's µC/TCP-IP stack and provide details on how and why those settings matter. The series will be geared towards users implementing TCP sockets and not UDP. However, much of the content in the first post is applicable to all projects based on µC/TCP-IP, regardless of the underlying transport protocol.
To make the most of this series of posts, you should keep the needs and objectives of your project in mind. The following will assist in navigating the two posts.
- Is system optimization a key part of your application's memory management or streamlining performance? The descriptions of cand net_cfg.h in Configuring uC/TCP-IP Part I will give you an informative look at key µC/TCP-IP configuration parameters.
- Is your application locking up after receiving data packets from another device? Part I’s content explains how the stack utilizes buffers to process packets.
- Does your network have high levels of traffic? The section about Stack Window Size configurations in Part II will elaborate on effectively setting these parameters.
- Are you developing a server that will service multiple clients? The Sockets Configuration section in Part 2 will help get you started.
- Are you looking to monitor and analyze the network traffic handled by your system? The TCP/IP Analysis Tools section in Part II will go over this in more detail.
Configuring µC/TCP-IP typically involves making changes to at least two files: net_dev_cfg.c and net_cfg.h. We examine key definitions in these files below.
net_dev_cfg.c (Device-Level Configuration File)
First-time users typically experience issues caused by incorrectly configuring the number of large and small buffers and the number of descriptors. If these are not given appropriate values relative to one another, they will cause problems within your application. Typically, these are the source of many problems in µC/TCP-IP-based projects, as they directly impact the memory usage and performance of the system.
const NET_DEV_ETHER_MCU NetDev_Cfg_MCU_Ether0 = {
NET_IF_MEM_TYPE_MAIN, /* Receive Buffer Memory Pool Type */
1536u, /* Large Rx Buffer Size (in Octets) */
12u, /* Number of Large Rx Buffers */
32u, /* Alignment of Receive Buffers (in Octets) */
0u, /* Desired offset from base receive… */
/* …index (if needed) */
NET_IF_MEM_TYPE_MAIN, /* Transmit Buffer Memory Pool Type */
1536u, /* Large Tx Buffer Size (in Octets) */
10u, /* Number of Large Tx Buffers */
60u, /* Small Tx Buffer Size (in Octets) */
10u, /* Number of Small Tx Buffers */
32u, /* Alignment of Transmit Buffers (in Octets) */
0u, /* Offset from base transmit index (if needed) */
0x00000000u, /* Base Address of Dedicated Memory… */
/* (if available) */
0u, /* Size of dedicated memory in octets… */
/* …(if needed) */
5u, /* Number of device receive descriptors */
10u, /* Number of device transmit descriptors */Shown above is a partial example of the network interface configuration struct declared in net_dev_cfg.c. We will now discuss these configurable fields in more detail.
Large Rx Buffer Size
When configuring your large receive buffers in µC/TCP-IP, you must consider the physical driver being used by your application. If there is a required alignment size, you must honor this. In the struct above, the large receive buffers are configured to 1536 octets(bytes). This follows from the maximum Ethernet frame size of 1518. Since the buffers for the example device must be aligned to 32 bytes and 1518 is not an even factor of 32, this value is bumped up to the nearest evenly aligned 32-bit value, 1536. Under most circumstances, the size of the large receive buffers should not be less than 1518 so that each buffer can handle the largest allowable Ethernet frame.
Number of Large Rx Buffers
A minimum of one large Rx buffer must be defined. Ideally, you would allocate enough large receive buffers in order to avoid all buffer resources being completely consumed. This prevents the creation of a deadlock situation or prematurely closing sockets due to dropped/missed TCP packets. The optimal setting for a system depends on a number of factors and requires some trial by the developer to find that "golden" value that works best. More on this soon.
Large Tx Buffer Size
For optimum throughput, this should be set as the maximum packet size (plus any additional space needed to meet the necessary alignment requirements). For reduced RAM usage, reduce the buffers' size but expect a performance hit (usually negatively). Ultimately, the performance impact of any change in the size of the large transmit buffers is determined by the profile of the network traffic generated by your application. If your system primarily sends data in small chunks, a decrease in the buffer size likely won’t reduce the overall performance. Conversely, if your application is written to perform large data transmissions, a decrease in the buffer size could have a substantial impact on the system’s throughput.
Number of Large Tx Buffers
As with the large Rx buffers, the optimal number of large Tx buffers depends on several key factors. Thus, your system may need to experiment a bit in order to find the ideal setting for best performance. In all cases, the total number of Tx buffers (large and small combined) must be at least one.
Small Tx Buffer Size
Use the defined variable NET_BUF_DATA_SIZE_MIN when setting up this configuration. This includes any additional bytes required to ensure this value properly aligns with your devices pre-set alignment. If configured larger than this, a deadlock situation could occur. Generally, small Tx buffers are utilized for transmission of packets without a data payload like TCP, ACK and RST segments. Small buffers are not recommended for use with transmitting data payloads by the application. To ensure this we recommend setting them to a size where only "payload-less" packets can utilize them. This may vary depending on your interface's size requirements but can be found in the hardware reference manual if not already known.
Number of Small Tx Buffers
Ideally, you want one small buffer for every two to four sockets being used. If performance is very important more small buffers can improve this, although the exact gains will differ according to the transmit behavior of the application.
Number of Rx Descriptors
Set this as 50% - 70% of the interface's number of receive buffers. If you have 1 interface and 10 Rx buffers you want at least 5 descriptors up to 7 ideally.
Number of Tx Descriptors
This should be equal to the total of all large and small transmit buffers configured for the interface.
Size of a Typical TCP Buffer
Descriptor chain with some descriptors currently processing buffers and one empty descriptor
Descriptors manage buffers. They indicate which buffers are available to receive incoming packets.
When a buffer pointed to by a descriptor is loaded with a packet it is then passed to the software
for processing, which is when the own bit is SW owned.
Own Bit used to indicate if the descriptor is presently owned by the hardware or software
System Resources: 3 Rx Descriptors – 5 Rx Buffers – 1 Interface
Buffer 1 and 2 received incoming packets. They are then passed onto the software
for processing. Buffer 3 is still HW owned and awaiting a packet. While
Buffer 1 and 2 are processed by the SW the descriptors can point to other empty
buffers to keep the queue going. This is one example why it's important to have
more buffers than descriptors in your system.
This illustrates how buffers 4 and 5 are queued up by the descriptors for incoming packets.
Buffer 1 and 2 continue to process while the descriptors continue queuing up the next
available buffers. Once Buffer 1 and 2 finish they will be available once again for new
incoming packets.
The above images illustrate how the receive buffers and descriptors that are configured via net_dev_cfg.c interact within the stack. If there are not enough free buffers to associate with descriptors while packets are processed, performance can suffer. This is the reason for the earlier recommendation to keep the number of receive descriptors at 50% to 70% of the number of buffers.
More documentation on the configurable parameters in net_dev_cfg.c can be found at the Micrium documentation’s page:
Network Interface Configuration
Your settings in net_dev_cfg.c will influence how you should configure the defines in net_cfg.h. We consider important settings from the latter file below.
net_cfg.h: (Higher-Level Configuration)
NET_CFG_IF_RX_Q_SIZE
Define this constant relative to the total amount of receive buffers your application has across all devices. If you’re using one interface this should be identical to the number of receive buffers defined in net_dev_cfg.c for that interface. With multiple interfaces, set this to be the same as the total number of receive buffers on across interfaces.
NET_CFG_IF_TX_DEALLOC_Q_SIZE
This constant should equal the total number of large and small transmit buffers defined across ALL interfaces in the application. If your system has one interface this configurable should be the sum of that interface's large and small transmit buffers. If you have two or more this should be the total of the buffers on each created interface within your system.
NET_IF_CFG_MAX_NBR_IF
Set this equal to the number of interface devices you plan to implement in your project. You do NOT need to define a new interface structure for each implementation. The definition in net_dev_cfg.c can be re-used for creating multiple instances with the same settings.
NET_SOCK_CFG_SOCK_NBR_TCP
This specifies how many TCP sockets can be created in a system. Each call to NetSock_Open() requires a socket from the corresponding TCP or UDP socket define and is used to determine how many total sockets can be expected at any given time. Closed sockets are free to be used with another NetSock_Open() call.
NET_SOCK_CFG_SOCK_NBR_UDP
This defines how many UDP sockets a system can contain. Similar overarching principles for defining TCP configuration apply to UDP except that the UDP version should account for NetSock_Open() calls to only UDP sockets.
NET_IPv4_CFG_IF_MAX_NBR_ADDR
This allows for configuring multiple IP addresses on one interface. It is useful for systems requiring multiple connections but lacking the physical resources to assign a one-to-one hardware to address setup. This configuration applies to IPv4.
NET_IPv6_CFG_IF_MAX_NBR_ADDR
Similar to the IPv4 configuration above but for IPv6.
NET_SOCK_CFG_CONN_ACCEPT_Q_SIZE_MAX
This definition establishes the maximum size of each sockets connection queue. This is used when remote devices when establishing multiple connections. Some requests may need to be queued while the initial requests are processed. When configuring a listen socket you must specify the connection queue size. The size indicates how many connections can be placed in the socket accept queue. This can be used to limit how many buffers a socket can consume to prevent sockets from using up all available buffer resources.
NET_SOCK_CFG_RX_Q_SIZE_OCTET
This represents the number of bytes a socket can queue up when data is received. This value should not exceed the cumulative amount of space provided by the system's buffers. This value should be fine-tuned for each individual application for the best performance possible. It can be modified, per socket, at run time with public API’s but cannot exceed this default value. This configuration acts as a load balancer between network devices. At a maximum value, high bandwidth applications would consume all available buffers preventing other applications from receiving anything. Setting this configuration is the equivalent of using SO_RECVBUF in sockets.
NET_SOCK_CFG_TX_Q_SIZE_OCTET
This value represents how many bytes can be queued up by a socket for transmission. Like the Rx configuration, it is a load balancer for UDP connections. µC/TCP-IP’s transmission function report’s back the total number of bytes transmitted by a completed transmit call. If the data to be sent exceeds the available buffer size, the stack will inform the application that xx Bytes out of yy Bytes were sent and another call to transmit must be done to finish the transmission. This configuration is the equivalent of using SO_SNDBUF in BSD sockets.
Optional: These definitions are considered "advanced," meaning that they are normally commented out. If you'd like to modify them, then simply remove the commenting.
NET_TCP_DFLT_RX_WIN_SIZE_OCTET
This defines the window size used during reception operations. The second part of this series will go more in-depth on that.
NET_TCP_DFLT_TX_WIN_SIZE_OCTET
This defines the window size used during any transmission operations. The second part of this series will go more in-depth on that.
Conclusion
Part I covered some of the configuration basics for µC/TCP-IP. It started with the driver level, reviewing configuration parameters contained in net_dev_cfg.c and examining how the buffers and descriptors, whose allocation is controlled by this file, interact. We then moved up to the stack level configured via net_cfg.h and reviewed some of the more commonly used definitions. The next post will tackle slightly more advanced configuration topics, such as TCP window size. We'll introduce a tool that can help µC/TCP-IP users monitor performance and confirm that the stack is configured as expected.
