The operating system emulation layer is located in two files, cc.h and sys_arch.c. It provides a common interface between the lwIP code and the underlying operating system kernel. The general idea is that porting lwIP to new architectures requires only small changes to a few header files and a new sys_arch implementation. It is also possible to do a sys_arch implementation that does not rely on any underlying operating system.

Sources of informationEdit

In addition to this page, you should probably have a look at the doc/sys_arch.txt file that is shipped along with the lwip source code. Looking at the ports/ directory of the contrib tarball can also help.


This is a basic header that describes your compiler and processor to lwIP. The following things must be defined:

  • Data types:
    • u8_t, u16_t, u32_t: The unsigned data types. For a 32-bit microprocessor, this is usually unsigned char, unsigned short, and unsigned int.
    • s8_t, s16_t, s32_t: The signed counterparts.
    • mem_ptr_t: A generic pointer type. It has to be an integer type (not void*, due to some pointer arithmetics).

  • printf formatters for data types:
    • U16_F, S16_F, X16_F, U32_F, S32_F, X32_F, SZT_F
    • usually defined to "hu", "d", "hx", "u", "d", "x", "uz"
    • These formatters are used in several lwip files, yet mainly for diagnostics (LWIP_PLATFORM_DIAG) and debug output (LWIP_DEBUGF), so you might not need them if you disable that output.


Packet headers data is stored in network byte order which is the the big-endian mode. If your processor architure is little-endian, then we need to convert data using htons()/htonl()/ntohs()/ntohl() functions.

If your processor supports both big-endian and little-endian mode or you are yet free to chose a processor, big-endian might be a better choice regarding performance, as host- and network-byte order are the same.

LwIP provides standard functions for this, but it's often more efficient to define your own macros or functions if your compiler provides intrinsics or assembly instructions for byte swapping.

For example:

#define LWIP_PLATFORM_HTONS(x) ( (((u16_t)(x))>>8) | (((x)&0xFF)<<8) )
#define LWIP_PLATFORM_HTONL(x) ( (((u32_t)(x))>>24) | (((x)&0xFF0000)>>8) \ 
                               | (((x)&0xFF00)<<8) | (((x)&0xFF)<<24) )
  • Computing cheksums:

IP protocols use checksums (see RFC 1071). LwIP gives you a choice of 3 algorithms:

  1. load byte by byte, construct 16 bits word and add: not efficient for most platforms
  2. load first byte if odd address, loop processing 16 bits words, add last byte.
  3. load first byte and word if not 4 byte aligned, loop processing 32 bits words, add last word/byte.

you may also use your own checksum routine (e.g. using assembly to add 32 bits words with carry)

u16_t my_chksum(void *dataptr, u16_t len);
#define LWIP_CHKSUM my_chksum
  • Structure packing:

LwIP accesses to 16 and 32 bits data fields in protocol headers which may be not aligned in memory. If your processor cannot read from/write to misaligned addresses, then you need to tell your compiler that the data may be not aligned and it must generate multiple byte or word load/stores to access it.

In all protocol structures, 16/32 bits values are 16 bits aligned, so choosing a 2 byte alignement for structures should be safe.

The common case is using Ethernet interfaces and MEM_ALIGNMENT=4. Ethernet header without VLAN is 14 bytes, so if you can/must have ETH_PAD_SIZE=2, then IP headers and higher layers are 4 bytes aligned and you may not need packing !

In any case, make sure the system runs stable when chosing a structure packing setting different to 1!

Usually a packed structure is created like that:

   #  include "arch/bpstruct.h"
   struct <structure_name> {
       PACK_STRUCT_FIELD(<type> <field>);
       PACK_STRUCT_FIELD(<type> <field>);
   #  include "arch/epstruct.h"

Appropriate definitions of PACK_STRUCT_BEGIN, PACK_STRUCT_FIELD, PACK_STRUCT_STRUCT and PACK_STRUCT_END should be included in cc.h. The required definitions mainly depend on your compiler. Only PACK_STRUCT_STRUCT is required, lwip/src/include/lwip/arch.h provides empty defaults for the others. Optionally you can also define PACK_STRUCT_USE_INCLUDES and provide bpstruct.h and epstruct.h.
  • Platform specific diagnostic output
    • LWIP_PLATFORM_DIAG(x) - non-fatal, print a message. Uses printf formating.
    • LWIP_PLATFORM_ASSERT(x) - fatal, print message and abandon execution. Uses printf formating. Unlike assert() from the standard c library, the parameter x is the message, not a condition.

Preemption protection Edit

Information about this can be found in doc/sys_arch.txt and src/include/lwip/sys.h. It is called "lightweight" synchronization mechanisms, as some implementations can provide a more light-weight protection mechanism than using semaphores.

The macros used in lwip are:

  • SYS_ARCH_PROTECT(x): Begin a block of protection (i.e. a section where task switching may not occur). Should generally return the previous state.
  • SYS_ARCH_UNPROTECT(x): End a block of protection, restoring the protection state back to "x".
  • SYS_ARCH_DECL_PROTECT(x): Declare the variable "x" to hold the protection. Used so you can specify what data type the protection will require.

It is possible that the parameter "x" is not even used (the unix port from contrib does not use it).

Currently (lwip version 1.4.0), the following modules use these macros[1]:

  • src/api/api_msg.c,
  • src/api/sockets.c,
  • src/core/mem.c,
  • src/core/memp.c,
  • src/core/pbuf.c and
  • src/core/netif.c.

Implementors have three options how to implement it:

  1. Provide no implementation. This is appropriate when lwip is not used within multiple threads and/or in interrupts.
    Then do not define SYS_ARCH_PROTECT in cc.h or SYS_LIGHTWEIGHT_PROT in lwipopts.h. So SYS_LIGHTWEIGHT_PROT defaults to 0 in opt.h and sys.h provides empty default definitions for SYS_ARCH_DECL_PROTECT, SYS_ARCH_PROTECT and SYS_ARCH_UNPROTECT.
  2. Provide an implementation in cc.h by defining the macros SYS_ARCH_DECL_PROTECT, SYS_ARCH_PROTECT and SYS_ARCH_UNPROTECT.
  3. RECOMMENDED: Provide an implementation in sys_arch.h and sys_arch.c. Do not define the macros in cc.h. Instead define SYS_LIGHTWEIGHT_PROT as 1 in lwipopts.h. Then sys.h will provide default definitions (as of version 1.3.0) of these macros as:
#define SYS_ARCH_DECL_PROTECT(lev) sys_prot_t lev
                #define SYS_ARCH_PROTECT(lev) lev = sys_arch_protect()
                #define SYS_ARCH_UNPROTECT(lev) sys_arch_unprotect(lev)
Define the lowercase versions in sys_arch.h (and sys_arch.c) (see that header file section below). In most cases, this is the "right" way to define them. lwIP uses only the UPPERCASE versions in its code.

sys_arch.c Edit

The sys_arch provides semaphores and mailboxes to lwIP. For the full lwIP functionality, multiple threads support can be implemented in the sys_arch, but this is not required for the basic lwIP functionality. Previous versions of lwIP required the sys_arch to implement timer scheduling as well but as of lwIP 0.5 this is implemented in a higher layer.

There has been some changes in this layer since 1.3.2, most prominently the functions returns an error value and created objects are returned via a pointer provided as an argument to the function.

Semaphores Edit

Semaphores can be either counting or binary - lwIP works with both kinds. Semaphores are represented by the type sys_sem_t which is typedef'd in the sys_arch.h file. lwIP does not place any restrictions on how sys_sem_t should be defined or represented internally, but typically it is a pointer to an operating system semaphore or a struct wrapper for an operating system semaphore. Warning! Semaphores are used in lwip's memory-management functions (mem_malloc(), mem_free(), etc.) and can thus not use these functions to create and destroy them.

The following functions must be defined:

  • err_t sys_sem_new(sys_sem_t *sem, u8_t count): Creates a new semaphore returns it through the sem pointer provided as argument to the function, in addition the function returns ERR_MEM if a new semaphore could not be created and ERR_OK if the semaphore was created. The count argument specifies the initial state of the semaphore.
  • void sys_sem_free(sys_sem_t * sem): Frees a semaphore created by sys_sem_new. Since these two functions provide the entry and exit point for all semaphores used by lwIP, you have great flexibility in how these are allocated and deallocated (for example, from the heap, a memory pool, a semaphore pool, etc).
  • void sys_sem_signal(sys_sem_t * sem): Signals (or releases) a semaphore referenced by * sem.
  • u32_t sys_arch_sem_wait(sys_sem_t * sem, u32_t timeout): Blocks the thread while waiting for the semaphore to be signaled. The timeout parameter specifies how many milliseconds the function should block before returning; if the function times out, it should return SYS_ARCH_TIMEOUT. If timeout=0, then the function should block indefinitely. If the function acquires the semaphore, it should return how many milliseconds expired while waiting for the semaphore. The function may return 0 if the semaphore was immediately available.

Note that there is another function sys_sem_wait in sys.c, but it is a wrapper for the sys_arch_sem_wait function. Please note that it is important for the semaphores to return an accurate count of elapsed milliseconds, since they are used to schedule timers in lwIP. See the timer section below for more information.

Mailboxes Edit

Mailboxes are used for message passing and can be implemented either as a queue which allows multiple messages to be posted to a mailbox, or as a rendez-vous point where only one message can be posted at a time. lwIP works with both kinds, but the former type will be more efficient. A message in a mailbox is just a pointer, nothing more.

Mailboxes are equivalently represented by the type sys_mbox_t. lwIP does not place any restrictions on how sys_mbox_t is represented internally, but it is typically a structure pointer type to a structure that wraps the OS-native mailbox type and its queue buffer.

The following functions must be defined:

  • err_t sys_mbox_new(mbox_t * mbox, int size): Tries to create a new mailbox and return it via the mbox pointer provided as argument to the function. Returns ERR_OK if a mailbox was created and ERR_MEM if the mailbox on error.
  • void sys_mbox_free(sys_mbox_t * mbox): Deallocates a mailbox. If there are messages still present in the mailbox when the mailbox is deallocated, it is an indication of a programming error in lwIP and the developer should be notified.
  • void sys_mbox_post(sys_mbox_t * mbox, void *msg): Posts the "msg" to the mailbox.
  • u32_t sys_arch_mbox_fetch(sys_mbox_t * mbox, void **msg, u32_t timeout). Blocks the thread until a message arrives in the mailbox, but does not block the thread longer than timeout milliseconds (similar to the sys_arch_sem_wait() function). If timeout=0, then the function should block indefinitely. The msg argument is a pointer to the message in the mailbox and may be NULL to indicate that the message should be dropped. This should return either SYS_ARCH_TIMEOUT or the number of milliseconds elapsed waiting for a message.
  • u32_t sys_arch_mbox_tryfetch(sys_mbox_t * mbox, void **msg): This is similar to sys_arch_mbox_fetch, however if a message is not present in the mailbox, it immediately returns with the code SYS_MBOX_EMPTY. On success 0 is returned with msg pointing to the message retrieved from the mailbox. If your implementation cannot support this functionality, a simple workaround (but inefficient) is #define sys_arch_mbox_tryfetch(mbox,msg) sys_arch_mbox_fetch(mbox,msg,1).
  • err_t sys_mbox_trypost(sys_mbox_t * mbox, void *msg). Tries to post a message to mbox by polling (no timeout). The function returns ERR_OK on success and ERR_MEM if it can't post at the moment.

lwIP timeouts Edit

lwIP timeouts has been removed since 1.3.2, the functionality still exists but has been moved out of the porting layer and into lwIP itself to make porting easier and more straightforward.

Threads Edit

Threads are not required for lwIP, although lwIP is written to use them efficiently. The following function will be used to instantiate a thread for lwIP:

  • sys_thread_t sys_thread_new(char *name, void (* thread)(void *arg), void *arg, int stacksize, int prio): name is the thread name. thread(arg) is the call made as the thread's entry point. stacksize is the recommanded stack size for this thread. prio is the priority that lwIP asks for. Stack size(s) and priority(ies) have to be are defined in lwipopts.h, and so are completely customizable for your system.


void sys_init(void). lwIP system initialization. This function is called before the any other sys_arch-function is called and is meant to be used to initialize anything that has to be up and running for the rest of the functions to work (for example to set up a pool of semaphores).

Final declarations for sys_arch.h Edit

  • sys_sem_t, sys_mbox_t, sys_thread_t all refer to the appropriate types, whether native OS versions or wrappers used to make them conform to what lwIP expects. Usualy these are typedefed directly to the systems corresponding types but if wrapper functions are used these may be just about anything. References to a list of pre-alloced objects is but one example.
  • SYS_MBOX_NULL, SYS_SEM_NULL, almost always set equal to NULL. These define what a failure to create the system type will return.

sys_arch.h is also where sys_sem_valid(sem), sys_sem_set_invalid(sem), sys_mbox_valid(mbox) and sys_mbox_set_invalid(mbox) macros are created.

  • sys_sem_valid(sem) verifies that a semaphore is valid and can be used.
  • sys_sem_set_invalid(sem) sets a semaphore to a value that will be interpreted as invalid. lwIP uses this macro to make sure an old reference can't be reused.
  • sys_mbox_valid(mbox) and sys_mbox_set_invalid(mbox) works precisely as their similes for semaphores but as creation may differ between mailboxes and semaphores they have their own criteria.

Preemption protection Edit

If you define your own UPPERCASE versions, as explained above in the cc.h section, then you do not need to define these).

It is recommended that you define the following functions (rather than the UPPERCASE versions) appropriately.

These functions are used to disable interrupts around crucial blocks of code that should not be interrupted -- usually done in order to preserve data integrity. In order to allow your code to nest protection blocks (for example a block of protection inside another block of protection, possibly in different functions), a parameter (shown as "x" here) is used to remember the prior protection state so that when a block ends the previous state is restored.

  • sys_prot_t sys_arch_protect(void): This optional function does a "fast" critical region protection and returns the previous protection level. This function is only called during very short critical regions. An embedded system which supports ISR-based drivers might want to implement this function by disabling interrupts. Task-based systems might want to implement this by using a mutex or disabling tasking. This function should support recursive calls from the same task or interrupt. In other words, sys_arch_protect() could be called while already protected. In that case the return value indicates that it is already protected.
  • void sys_arch_unprotect(sys_prot_t pval): This optional function does a "fast" set of critical region protection to the value specified by pval. See the documentation for sys_arch_protect() for more information. This function is only required if your port is supporting an operating system.
  • Define the sys_prot_t type to reflect what these two functions need to operate on.

The following is an example of how these functions will be used in the default implementation.

lwIP code example Compiler expansion for default defintion
   /* My delicate block of code */
sys_prot_t lev;
   lev = sys_arch_protect();
   /* My delicate block of code */

Final thoughts Edit

Be careful with using mem_malloc() in sys_arch. Specifically, when mem_init() is run (in mem.c), it allocates a semaphore to block concurrent access in its routines. If sys_sem_new() calls mem_malloc to allocate its semaphore, then it will be calling mem_malloc() before the mem.c system has been initialized!

Footnotes Edit

  1. src$ find -type f -exec grep -l SYS_ARCH_PROTECT {} \; finds them and src/include/lwip/sys.h
Community content is available under CC-BY-SA unless otherwise noted.