buffer pools for network interfaces
, const char *name
, uint32_t h
Mbuf pools are intended to help drivers for interface cards that need huge
amounts of receive buffers, and additionally provides a mapping between these
buffers and 32-bit handles.
An example of these cards are the Fore/Marconi ForeRunnerHE cards. These employ
up to 8 receive groups, each with two buffer pools, each of which can contain
up to 8192. This gives a total maximum number of more than 100000 buffers.
Even with a more moderate configuration the card eats several thousand
buffers. Each of these buffers must be mapped for DMA. While for machines
without an IOMMU and with lesser than 4GByte memory this is not a problem, for
other machines this may quickly eat up all available IOMMU address space
and/or bounce buffers. On sparc64, the default I/O page size is 16k, so
mapping a simple mbuf wastes 31/32 of the address space.
Another problem with most of these cards is that they support putting a 32-bit
handle into the buffer descriptor together with the physical address. This
handle is reflected back to the driver when the buffer is filled, and assists
the driver in finding the buffer in host memory. For 32-bit machines, the
virtual address of the buffer is usually used as the handle. This does not
work for 64-bit machines for obvious reasons, so a mapping is needed between
these handles and the buffers. This mapping should be possible without
searching lists and the like.
An mbuf pool overcomes both problems by allocating DMA-able memory page wise
with a per-pool configurable page size. Each page is divided into a number of
equally-sized chunks, the last
of which are used by
the pool code (4 bytes). The rest of each chunk is usable as a buffer. There
is a per-pool limit on pages that will be allocated.
Additionally, the code manages two flags for each buffer:
“on-card” and “used”. A buffer may be in one of
- None of the flags is set.
- Both flags are set. The buffer is assumed to be handed over to the card
and waiting to be filled.
- The buffer was returned by the card and is now travelling through the
A pool is created with
call specifies a DMA tag dmat
to be used to
create and map the memory pages via
includes the pool overhead. It
means that to get buffers for 5 ATM cells (240 bytes), a chunk size of 256
should be specified. This results in 12 unused bytes between the buffer, and
the pool overhead of four byte. The total maximum number of buffers in a pool
). The maximum value for
is 2^14-1 (16383) and the maximum
is 2^9 (512). If the call is
successful, a pointer to a newly allocated struct
is set into the variable pointed to by
A pool is destroyed with
frees all pages and the pool structure itself. If compiled with
, the code checks that all
buffers are free. If not, a warning message is issued to the console.
A buffer is allocated with
returns the virtual address of the buffer and stores the physical address into
the variable pointed to by pa
. The handle is
stored into the variable pointed to by hp
The two most significant bits and the 7 least significant bits of the handle
are unused by the pool code and may be used by the caller. These are
automatically stripped when passing a handle to one of the other functions. If
a buffer cannot be allocated (either because the maximum number of pages is
reached, no memory is available or the memory cannot be mapped),
is returned. If a buffer could be
allocated, it is in the “on-card” state.
When the buffer is returned by the card, the driver calls
() with the handle. This function
returns the virtual address of the buffer and clears the
“on-card” bit. The buffer is now in the “used”
state. The function
() in that it does not clear
the “on-card” bit. This can be used for buffers that are
returned “partially” by the card.
A buffer is freed by calling
the virtual address of the buffer. This clears the “used” bit,
and puts the buffer on the free list of the pool. Note that free buffers are
NOT returned to the system. The function
() can be given to
() as the free function. The user
argument must be the pointer to the pool.
Before using the contents of a buffer returned by the card, the driver must call
() with the appropriate parameters.
This results in a call to bus_dmamap_sync(9)
All buffers in the pool that are currently in the “on-card” state
can be freed with a call to
(). This may be called by the
driver when it stops the interface. Buffers in the “used” state
are not freed by this call.
For debugging it is possible to call
(). This returns the number of
buffers in the “used” and “on-card” states and the
number of buffers on the free list.
() is currently a no-op
is missing the offset
and length parameters.