CnuasSwitch, Datasheet
| Item |
Value |
| Part |
cnuas-vswitchd |
| Type |
Virtual top-of-rack (TOR) switch |
| Version |
v0.1.0-8-gd630c1f |
| Repo |
PacketFive/CnuasSwitch |
1. Overview
CnuasSwitch is the emulated top-of-rack switch of the Cnuas fabric. The
cnuas-vswitchd daemon presents a 10-port switch that forwards both RoCEv2
(UDP/4791 over emulated Ethernet) and native InfiniBand (LRH/BTH) traffic
between attached CnuasNIC endpoints. It includes an
in-switch Subnet Manager (SM) and Subnet Management Agent (SMA), so an
InfiniBand fabric becomes operational without an external opensm.
Key features
- 10-port fixed switch: 8 fabric ports + uplink + console/observability.
- Dual-personality forwarding: RoCEv2 and native InfiniBand on the fabric ports.
- Per-port personality: RoCE, InfiniBand, hybrid, or stacked.
- Data Center Bridging: PFC (802.1Qbb), ETS (802.1Qaz), ECN (RFC 3168).
- Ethernet forwarding database (FDB) and InfiniBand Linear Forwarding Table (LFT).
- Built-in Subnet Manager with directed-route SMPs, LID assignment, and sweeps.
- Single-threaded
epoll event loop for deterministic, teachable behaviour.
- JSON management interface with 18+ commands; OpenTelemetry export.
2. Port model
flowchart TB
subgraph SW["cnuas-vswitchd (10 ports)"]
direction TB
subgraph FAB["Fabric ports 0-7 (RoCEv2 + InfiniBand)"]
direction LR
P0["port 0"]
P1["port 1"]
P2["port ..."]
P7["port 7"]
end
UP["port 8<br/>uplink (Ethernet)"]
CON["port 9<br/>console / OTel"]
CORE["Forwarding core<br/>classifier + FDB + LFT + DCB"]
SM["Subnet Manager<br/>ib_sm.c + ib_sma.c"]
MGMT["Mgmt API<br/>JSON / UNIX socket"]
FAB --- CORE
UP --- CORE
CON --- CORE
CORE --- SM
CORE --- MGMT
end
classDef p fill:#dbeafe,stroke:#1e40af,color:#1e3a8a
classDef c fill:#fef3c7,stroke:#b45309,color:#78350f
class P0,P1,P2,P7,UP,CON p
class CORE,SM,MGMT c
3. Functional specifications
| Parameter |
Value |
| Total ports |
10 (CNUAS_TOTAL_PORTS) |
| Fabric ports (RoCEv2 + IB) |
8 (ports 0, 7) |
| Uplink port |
1 (port 8, Ethernet) |
| Console / OTel port |
1 (port 9, observability) |
| Per-port personality |
RoCE / InfiniBand / hybrid / stacked |
| Supported protocols |
RoCEv2 (UDP dst 4791), native InfiniBand |
| Max frame / path MTU |
Up to 9216 B (jumbo); IB path MTU up to 4096 B |
| DCB features |
PFC (IEEE 802.1Qbb), ETS (IEEE 802.1Qaz), ECN (RFC 3168) |
| Ethernet forwarding |
MAC FDB (add/del/dump) |
| InfiniBand forwarding |
LID Forwarding Table (LFT), ~256 entries |
| LID addressing |
16-bit LIDs |
| Subnet Manager |
In-switch SM + SMA (GET/SET, directed-route SMPs, sweeps) |
| Concurrency model |
Single-threaded epoll loop, MAX_EVENTS = 32, 1000 ms tick |
4. Daemon internals
| Subsystem |
Source |
Responsibility |
| Event loop |
epoll_loop.c |
epoll_loop_init() / epoll_loop_run(); per-port and mgmt FDs |
| Management |
mgmt.c |
JSON command parse/dispatch, responses |
| Subnet Manager |
ib_sm.c |
ib_sm_init(), ib_sm_trigger(), sm_build_dr_smp(), sweeps |
| Subnet Mgmt Agent |
ib_sma.c |
sma_set_node_info/port_info/switch_info/pkey_table/guid_info |
| Telemetry |
otel.c |
OpenTelemetry counter export |
Four file descriptor context tags drive the epoll loop: FD_PORT_LISTENER,
FD_PORT_DATA, FD_MGMT_LISTENER, and FD_MGMT_CLIENT.
Each port carries its own state, holding the MAC address, personality and mode,
link state, VLAN, priority flow control queues, enhanced transmission selection
configuration, explicit congestion notification configuration, and counters for
frames and bytes in each direction, drops, pause frames sent and received, and
frames marked for congestion.
5. Management command set (JSON)
| Group |
Commands |
| Port |
port_status, set_port_mode, set_link_state |
| QoS / DCB |
set_pfc, set_ecn, set_ets |
| Ethernet FDB |
fdb_add, fdb_del, fdb_dump |
| InfiniBand LFT |
lft_add, lft_del, lft_dump |
| Subnet Manager |
sm_enable, sm_disable, sm_trigger, sm_status |
| Telemetry |
telemetry_dump, telemetry_clear |
Response format: {"status":"ok"|"error", ...}.
6. Interfaces
| Interface |
Description |
| Port transport |
UNIX domain sockets (SOCK_SEQPACKET), one per port |
| Management |
JSON over UNIX socket at /var/run/cnuas/mgmt.sock |
| Run directory |
/var/run/cnuas/ |
| Observability |
OpenTelemetry export on console port |
7. Software support
| Item |
Value |
| Daemon |
cnuas-vswitchd (C / GNU C) |
| Management CLI |
cnuas-cli (port status, SM, DCB configuration) |
| Dashboard |
cnuas-webui |
| Design reference |
Virtual Switch Design |
8. Typical usage
# Run the switch daemon
./build/cnuas-vswitchd --run-dir /var/run/cnuas --foreground
# Inspect ports and drive the Subnet Manager
cnuas-cli port status
cnuas-cli sm trigger
| Item |
Value |
| Repo |
PacketFive/CnuasSwitch |
| Submodule path |
src/cnuasswitch (in PacketFive/cnuas) |
| Version |
v0.1.0 |
| Language |
C (GNU C) |
| Build |
Make / Bazel |
10. Revision history
| Revision |
Date |
Notes |
| A |
2026-07-05 |
Initial datasheet |
| B |
2026-07-05 |
Added port diagram, daemon internals, full JSON command set (through I5e/I5f SM) |