PacketFive | Cnuas Virtual AI/HPC Infrastructure Cnuas Verbs Provider, Datasheet rdma-core user space verbs provider for the CnuasNIC adapter Document DS-CNU-014 Revision A Issued 03 August 2026 Status Published ============================================================================== Item Value --------------- ------------------------------- Part libcnuas-rdmav34.so Type rdma-core user space provider Provider name cnuas Package cnuas-libcnuas-provider Version v0.1.0-26-gbc3a3c9 Repo PacketFive/CnuasNIC 1. Overview The Cnuas verbs provider is the user space half of the RDMA stack. It plugs into the standard rdma-core library so that libibverbs applications, perftest, rdma_cm users and NCCL reach a CnuasNIC through the same calls they use for physical adapters. No application changes and no LD_PRELOAD shims are involved: rdma-core discovers the provider from a driver description file at startup and matches it to the cnuas_ib kernel device. Key features - Discovered by stock rdma-core, no patched libibverbs required. - Complete queue pair, completion queue and memory region life cycle. - Shared receive queue and multicast attachment. - Address handle creation for datagram traffic. - Versioned kernel to user ABI, refused on mismatch. 2. Discovery and loading Item Value ------------------------- ---------------------------------------------------------------- Driver description file cnuas.driver, containing driver cnuas Installed location The rdma-core provider directory, typically /etc/libibverbs.d/ Shared object libcnuas-rdmav34.so Registration macro PROVIDER_DRIVER(cnuas, cnuas_dev_ops) Kernel ABI version 1, accepted as both minimum and maximum Kernel counterpart cnuas_ib.ko, see Kernel Modules The rdmav34 suffix follows the rdma-core provider naming convention and records the provider ABI generation the library is built for. ABI MISMATCH IS A REFUSAL, NOT A FAULT The provider declares the same value as its minimum and maximum accepted kernel ABI. If the loaded cnuas_ib.ko reports a different version the provider declines the device and ibv_devinfo shows no adapter, rather than binding and failing later in an unclear way. Keep the module and the provider from the same release. 3. Verbs implemented Group Operations ---------------------- --------------------------------------------------------------- Device and port query_device_ex, query_port Protection domain alloc_pd, dealloc_pd Memory region reg_mr, dereg_mr Completion queue create_cq, destroy_cq, poll_cq, req_notify_cq Queue pair create_qp, modify_qp, destroy_qp Data path post_send, post_recv Shared receive queue create_srq, modify_srq, query_srq, destroy_srq, post_srq_recv Address handle create_ah, destroy_ah Multicast attach_mcast, detach_mcast Device limits that applications read through query_device_ex come from the kernel module and are tabulated in Kernel Modules section 4.1. 4. Application compatibility Consumer Status -------------------------------------------------- ---------------------------------------------------------------- ibv_devinfo, ibv_devices Works ib_send_bw, ib_write_bw, ib_read_bw, ib_send_lat Works rdma and ip tooling Works NCCL over the network transport Works, see tests/test_nccl_link.py opensm Not supported, no user space management datagram device exists Vendor specific verbs extensions Not supported 5. Validation Feature Test -------------------------------------------------- -------------------------- RoCEv2 end to end, two guests through the switch tests/test_rocev2_e2e.py Native InfiniBand end to end tests/test_ib_e2e.py Collective transport over the fabric tests/test_nccl_link.py These are integration tests: a meaningful run needs the switch daemon, two guest virtual machines and the provider at once, which is why they live in the superproject rather than in PacketFive/CnuasNIC. Current results are recorded in the Validation Matrix. 6. Integration information Item Value ---------------- --------------------------------------------------------------------------------------------------- Repo PacketFive/CnuasNIC Submodule path src/cnuasnic Source src/cnuasnic/userspace/, src/cnuasnic/rdma-core/ Build Make for the standalone object, CMake through rdma_provider() when built inside an rdma-core tree Debian package cnuas-libcnuas-provider Depends on libibverbs, cnuas_ib.ko Language C 7. Revision history Revision Notes ---------- ------------------------------------------------------------------------------------------ A First publication. Operation list read from the provider operations table in the source. ============================================================================== PacketFive, Packet Five Networks Ltd., Dublin, Ireland. Cnuas Virtual AI/HPC Infrastructure is published at https://github.com/PacketFive/cnuas under the Apache License 2.0; read it at https://github.com/PacketFive/cnuas/blob/main/LICENSE. Cnuas Virtual AI/HPC Infrastructure is emulation software. It is not affiliated with, endorsed by, or derived from any hardware vendor, and every device it models is a software artefact. The work is published by its authors in a personal capacity and is not sponsored or endorsed by any employer. Specifications describe the referenced revision of the software and may change without notice. Contact info@packetfive.com.