English

Netmem Support for Network Drivers

This document outlines the requirements for network drivers to support netmem, an abstract memory type that enables features like device memory TCP. By supporting netmem, drivers can work with various underlying memory types with little to no modification.

Benefits of Netmem :

  • Flexibility: Netmem can be backed by different memory types (e.g., struct page, DMA-buf), allowing drivers to support various use cases such as device memory TCP.

  • Future-proof: Drivers with netmem support are ready for upcoming features that rely on it.

  • Simplified Development: Drivers interact with a consistent API, regardless of the underlying memory implementation.

Design Principles

Memory providers (or the default page_pool allocator) allocate underlying memory (struct net_iov or struct page), cast it to netmem_ref, and supply it to page_pool. The page_pool, drivers, and networking stack operate on netmem_ref as the abstract type. Existing page_pool APIs that allocate or free struct page are legacy compatibility wrappers for drivers that do not yet support netmem_ref. Code that is not yet netmem-aware should be converted to netmem_ref unless it will never need to support netmem.

  1. Operate on netmem_ref, do not downcast: page_pool, drivers, and the core networking stack should deal with netmem_ref rather than struct net_iov or struct page. Downcasting netmem_ref to struct net_iov or struct page is not allowed unless a code path strictly cannot function without knowing the underlying memory type (for example, kmap_local_page()). In those cases, to keep call sites simple, add a netmem helper that performs the operation on behalf of the caller, cleanly handles all net_iov and page cases, and returns an error if the netmem type cannot support the requested operation.

  2. Decouple memory providers from net_iov: Memory providers are not architecturally limited to struct net_iov; a memory provider that returns struct page-backed netmem_refs to upper layers is allowed. Today, in-tree memory providers only supply struct net_iov and some existing code still reflects that limitation, but new code must not assume that using a memory provider implies net_iov memory and should, as much as possible, generalize existing limitations to match the design principles.

  3. Decouple net_iov from unreadability: struct net_iov is flexible and has no inherent restrictions; it may represent either CPU-readable or unreadable memory. Today, in-tree net_iov implementations are unreadable by the CPU (netmem_address() returns NULL) and some existing code still reflects that limitation, but new code must not assume net_iov implies unreadable memory (check readability via netmem_address() or skb_frags_readable() instead) and should, as much as possible, generalize existing limitations to match the design principles.

  4. Delegate complexity to the lowest layer: Each layer must respect its abstraction boundary. page_pool must not implement per-memory-provider custom logic in its main code; instead, it delegates provider-specific handling to struct memory_provider_ops. Similarly, core networking code should avoid per-netmem-type branching and instead delegate operations to netmem helpers that handle the underlying memory type.

  5. Do not mix skb fragment memory sources: All frags[] in an sk_buff must either be struct page-backed or belong to the same memory provider instance (for example, the same devmem binding or io_uring area). Mixing struct page and memory-provider fragments, or mixing fragments from different memory providers within a single sk_buff, is not allowed (including when coalescing sk_buffs).

Driver RX Requirements

  1. The driver must support page_pool.

  2. The driver must support the tcp-data-split ethtool option.

  3. The driver must use the page_pool netmem APIs for payload memory. The netmem APIs currently 1-to-1 correspond with page APIs. Conversion to netmem should be achievable by switching the page APIs to netmem APIs and tracking memory via netmem_refs in the driver rather than struct page * :

    • page_pool_alloc -> page_pool_alloc_netmem

    • page_pool_get_dma_addr -> page_pool_get_dma_addr_netmem

    • page_pool_put_page -> page_pool_put_netmem

    Not all page APIs have netmem equivalents at the moment. If your driver relies on a missing netmem API, feel free to add and propose to netdev@, or reach out to the maintainers and/or almasrymina@google.com for help adding the netmem API.

  4. The driver must use the following PP_FLAGS:

    • PP_FLAG_DMA_MAP: netmem is not dma-mappable by the driver. The driver must delegate the dma mapping to the page_pool, which knows when dma-mapping is (or is not) appropriate.

    • PP_FLAG_DMA_SYNC_DEV: netmem dma addr is not necessarily dma-syncable by the driver. The driver must delegate the dma syncing to the page_pool, which knows when dma-syncing is (or is not) appropriate.

    • PP_FLAG_ALLOW_UNREADABLE_NETMEM. The driver must specify this flag iff tcp-data-split is enabled.

  5. The driver must not assume the netmem is readable and/or backed by pages. The netmem returned by the page_pool may be unreadable, in which case netmem_address() will return NULL. The driver must correctly handle unreadable netmem, i.e. don’t attempt to handle its contents when netmem_address() is NULL.

    Ideally, drivers should not have to check the underlying netmem type via helpers like netmem_is_net_iov() or convert the netmem to any of its underlying types via netmem_to_page() or netmem_to_net_iov(). In most cases, netmem or page_pool helpers that abstract this complexity are provided (and more can be added).

  6. The driver must use page_pool_dma_sync_netmem_for_cpu() in lieu of dma_sync_single_range_for_cpu(). For some memory providers, dma_syncing for CPU will be done by the page_pool, for others (particularly dmabuf memory provider), dma syncing for CPU is the responsibility of the userspace using dmabuf APIs. The driver must delegate the entire dma-syncing operation to the page_pool which will do it correctly.

  7. Avoid implementing driver-specific recycling on top of the page_pool. Drivers cannot hold onto a struct page to do their own recycling as the netmem may not be backed by a struct page. However, you may hold onto a page_pool reference with page_pool_fragment_netmem() or page_pool_ref_netmem() for that purpose, but be mindful that some netmem types might have longer circulation times, such as when userspace holds a reference in zerocopy scenarios.

Driver TX Requirements

  1. The Driver must not pass the netmem dma_addr to any of the dma-mapping APIs directly. This is because netmem dma_addrs may come from a source like dma-buf that is not compatible with the dma-mapping APIs.

    Helpers like netmem_dma_unmap_page_attrs() & netmem_dma_unmap_addr_set() should be used in lieu of dma_unmap_page[_attrs](), dma_unmap_addr_set(). The netmem variants will handle netmem dma_addrs correctly regardless of the source, delegating to the dma-mapping APIs when appropriate.

    Not all dma-mapping APIs have netmem equivalents at the moment. If your driver relies on a missing netmem API, feel free to add and propose to netdev@, or reach out to the maintainers and/or almasrymina@google.com for help adding the netmem API.

  2. Driver should declare support by setting netdev->netmem_tx to the appropriate mode:

    • NETMEM_TX_DMA: for physical devices that perform DMA.

    • NETMEM_TX_NO_DMA: for virtual or passthrough devices that do not DMA, but still support handling of netmem-backed skbs.