HiSwitch, Datasheet
Part: hicain-vswitchd ·
Type: Virtual top-of-rack (TOR) switch ·
Version: `v0.1.0-4-g1700b86` ·
Repo: PacketFive/HiSwitch
1. Overview
HiSwitch is the emulated top-of-rack switch of the HiCAIN fabric. The
hicain-vswitchd daemon presents a 10-port switch that forwards both RoCEv2
(UDP/4791 over emulated Ethernet) and native InfiniBand (LRH/BTH) traffic
between attached HiNIC endpoints. It includes an
in-switch Subnet Manager (SM) and Subnet Management Agent (SMA), so an
InfiniBand fabric becomes operational without an external opensm.
Key features
- 10-port fixed switch: 8 fabric ports + uplink + console/observability.
- Dual-personality forwarding: RoCEv2 and native InfiniBand on the fabric ports.
- Per-port personality: RoCE, InfiniBand, hybrid, or stacked.
- Data Center Bridging: PFC (802.1Qbb), ETS (802.1Qaz), ECN (RFC 3168).
- Ethernet forwarding database (FDB) and InfiniBand Linear Forwarding Table (LFT).
- Built-in Subnet Manager with directed-route SMPs, LID assignment, and sweeps.
- Single-threaded
epoll event loop for deterministic, teachable behaviour.
- JSON management interface with 18+ commands; OpenTelemetry export.
2. Port model
flowchart TB
subgraph SW["hicain-vswitchd (10 ports)"]
direction TB
subgraph FAB["Fabric ports 0-7 (RoCEv2 + InfiniBand)"]
direction LR
P0["port 0"]
P1["port 1"]
P2["port ..."]
P7["port 7"]
end
UP["port 8<br/>uplink (Ethernet)"]
CON["port 9<br/>console / OTel"]
CORE["Forwarding core<br/>classifier + FDB + LFT + DCB"]
SM["Subnet Manager<br/>ib_sm.c + ib_sma.c"]
MGMT["Mgmt API<br/>JSON / UNIX socket"]
FAB --- CORE
UP --- CORE
CON --- CORE
CORE --- SM
CORE --- MGMT
end
classDef p fill:#dbeafe,stroke:#1e40af,color:#1e3a8a
classDef c fill:#fef3c7,stroke:#b45309,color:#78350f
class P0,P1,P2,P7,UP,CON p
class CORE,SM,MGMT c
3. Functional specifications
| Parameter |
Value |
| Total ports |
10 (HICAIN_TOTAL_PORTS) |
| Fabric ports (RoCEv2 + IB) |
8 (ports 0, 7) |
| Uplink port |
1 (port 8, Ethernet) |
| Console / OTel port |
1 (port 9, observability) |
| Per-port personality |
RoCE / InfiniBand / hybrid / stacked |
| Supported protocols |
RoCEv2 (UDP dst 4791), native InfiniBand |
| Max frame / path MTU |
Up to 9216 B (jumbo); IB path MTU up to 4096 B |
| DCB features |
PFC (IEEE 802.1Qbb), ETS (IEEE 802.1Qaz), ECN (RFC 3168) |
| Ethernet forwarding |
MAC FDB (add/del/dump) |
| InfiniBand forwarding |
LID Forwarding Table (LFT), ~256 entries |
| LID addressing |
16-bit LIDs |
| Subnet Manager |
In-switch SM + SMA (GET/SET, directed-route SMPs, sweeps) |
| Concurrency model |
Single-threaded epoll loop, MAX_EVENTS = 32, 1000 ms tick |
4. Daemon internals
| Subsystem |
Source |
Responsibility |
| Event loop |
epoll_loop.c |
epoll_loop_init() / epoll_loop_run(); per-port and mgmt FDs |
| Management |
mgmt.c |
JSON command parse/dispatch, responses |
| Subnet Manager |
ib_sm.c |
ib_sm_init(), ib_sm_trigger(), sm_build_dr_smp(), sweeps |
| Subnet Mgmt Agent |
ib_sma.c |
sma_set_node_info/port_info/switch_info/pkey_table/guid_info |
| Telemetry |
otel.c |
OpenTelemetry counter export |
epoll FD context tags: FD_PORT_LISTENER, FD_PORT_DATA,
FD_MGMT_LISTENER, FD_MGMT_CLIENT.
Per-port state: MAC, personality/mode, link state, VLAN, PFC queues, ETS
config, ECN config, and counters (rx/tx frames/bytes, drops, PFC pause
sent/received, ECN marked).
5. Management command set (JSON)
| Group |
Commands |
| Port |
port_status, set_port_mode, set_link_state |
| QoS / DCB |
set_pfc, set_ecn, set_ets |
| Ethernet FDB |
fdb_add, fdb_del, fdb_dump |
| InfiniBand LFT |
lft_add, lft_del, lft_dump |
| Subnet Manager |
sm_enable, sm_disable, sm_trigger, sm_status |
| Telemetry |
telemetry_dump, telemetry_clear |
Response format: {"status":"ok"|"error", ...}.
6. Interfaces
| Interface |
Description |
| Port transport |
UNIX domain sockets (SOCK_SEQPACKET), one per port |
| Management |
JSON over UNIX socket at /var/run/hicain/mgmt.sock |
| Run directory |
/var/run/hicain/ |
| Observability |
OpenTelemetry export on console port |
7. Software support
| Item |
Value |
| Daemon |
hicain-vswitchd (C / GNU C) |
| Management CLI |
hicain-cli (port status, SM, DCB configuration) |
| Dashboard |
hicain-webui |
| Design reference |
Virtual Switch Design |
8. Typical usage
# Run the switch daemon
./build/hicain-vswitchd --run-dir /var/run/hicain --foreground
# Inspect ports and drive the Subnet Manager
hicain-cli port status
hicain-cli sm trigger
| Item |
Value |
| Repo |
PacketFive/HiSwitch |
| Submodule path |
src/hiswitch (in PacketFive/vdc) |
| Version |
v0.1.0 |
| Language |
C (GNU C) |
| Build |
Make / Bazel |
10. Revision history
| Revision |
Date |
Notes |
| A |
2026-07-05 |
Initial datasheet |
| B |
2026-07-05 |
Added port diagram, daemon internals, full JSON command set (through I5e/I5f SM) |