Cnuas Minimum System Requirements¶
Cnuas is a software emulation of an ORV3 rack-scale AI and HPC accelerator platform. Nothing in it needs a GPU, an accelerator card, a smart NIC or a baseboard management controller. What it does need is host RAM, host disk and a reasonably recent Linux.
This page states what a machine must have for each way Cnuas can be used. The tiers are cumulative. A machine that meets the rack-scale tier also meets the host-only tier.
Sizes here are measured on the reference host rather than estimated. Where a figure depends on a setting the setting is named, so the number can be recomputed rather than trusted.
1. Choosing a tier¶
| Tier | What it runs | Guest kernel needed |
|---|---|---|
| A. Host only | Soft-GPU, soft switch, control plane, facility twin | No |
| B. Single VM | One emulated sled with CnuasGPU and CnuasNIC | Yes |
| C. Rack scale | Two sleds, fabric, ORV3 power shelf, OpenBMC | Yes |
| D. Full build | Building QEMU, the guest kernel and the golden image | Yes |
| E. Many racks | More racks than one machine carries, across a fleet | Yes |
Most users who want to run a model on the Soft-GPU, or to drive the campus twin, need only tier A. Tier A needs no virtualisation support, no KVM and no root.
2. Tier A, host only¶
| Resource | Minimum | Recommended |
|---|---|---|
| CPU | x86-64 with SSE2 | x86-64 with AVX2 or AVX-512 |
| Cores | 2 | 8 |
| RAM | 2 GiB plus the device arena | 16 GiB |
| Disk | 3 GiB | 10 GiB |
| OS | Linux with glibc 2.31 or newer | Ubuntu 24.04 |
| Python | 3.11 | 3.12 |
The Soft-GPU allocates its device memory as a single host mapping. The default
is 512 MiB and the size is set with CNUAS_HOST_ARENA_MB, so budget host RAM
for the arena on top of the base figure. A 32 GiB arena for a large model needs
a host that can map 32 GiB.
The arena is mapped with MAP_NORESERVE, so it is address space rather than
committed memory and pages are faulted in as the model touches them. This is
why an arena larger than host RAM can be declared. It is also why declaring one
is unwise. If the working set exceeds RAM the host swaps.
Accepted arena sizes run from 1 MiB to 1048576 MiB, which is 1 TiB. A value outside that range, or one that does not parse, prints a notice on stderr and falls back to the 512 MiB default rather than failing silently.
AVX-512 is used when present and is not required. The host backend selects its kernels at run time, so the same binary runs on a machine without it.
3. Tier B, one emulated sled¶
Add to tier A.
| Resource | Minimum | Recommended |
|---|---|---|
| Virtualisation | Intel VT-x or AMD-V, with /dev/kvm readable |
Same |
| RAM | Guest RAM plus GPU device memory plus 2 GiB | 32 GiB |
| Disk | 30 GiB | 60 GiB |
The emulated CnuasGPU backs BAR1 with host RAM, so the host must carry the guest RAM and the device memory at the same time. The tooling default is a 4 GiB guest with 4 vCPUs.
| Guest RAM | devmem_size |
Host RAM to budget |
|---|---|---|
| 4 GiB | 256 MiB (default) | about 7 GiB |
| 4 GiB | 8 GiB | about 14 GiB |
| 8 GiB | 16 GiB | about 26 GiB |
| 16 GiB | 64 GiB | about 82 GiB |
QEMU faults guest and device pages in lazily, so resident memory starts far below these figures and grows with use. Size the host for the working set, not for the declared size, but do not declare a size the host could never reach.
devmem_size accepts 16 MiB to 1 TiB and must be a power of two. It is carried
on a 64-bit prefetchable BAR, which is what allows sizes at or above 4 GiB to be
mapped at all. Values outside the range, or values that are not a power of two,
fail at device realise with a message rather than producing a device the guest
cannot map.
Without KVM the guest still boots under TCG. Expect it to be slow. TCG is supported for machines that cannot expose virtualisation, such as a nested cloud instance, and is not the intended path.
4. Tier C, rack scale¶
Add to tier B.
| Resource | Minimum | Recommended |
|---|---|---|
| Cores | 8 | 16 |
| RAM | 16 GiB | 64 GiB |
| Disk | 60 GiB | 120 GiB |
A two-sled rack runs two guest VMs, the virtual switch, the CnuasLink fabric, the OpenBMC rack management controller on an emulated AST2600, and twelve Renode satellites for the ORV3 power shelf. The twelve satellites are Cortex-M machines and are individually cheap. The two guests dominate.
Renode is needed only for the ORV3 power shelf. It runs on Mono or .NET and adds about 1 GiB of RAM and 500 MiB of disk.
The reference host used for validation has 62 GiB of RAM, 7 GiB of swap and AVX2 without AVX-512. It runs tier C comfortably.
5. Tier D, building from source¶
Add to tier C.
| Resource | Minimum | Recommended |
|---|---|---|
| Cores | 8 | 16 or more |
| RAM | 16 GiB | 32 GiB |
| Disk | 120 GiB free | 200 GiB free |
Measured sizes of a full checkout with submodules and a completed build.
| Tree | Size |
|---|---|
.git including submodules |
about 7.0 GiB |
qemu source and build |
about 2.3 GiB |
linux source and build |
about 1.7 GiB |
out build products |
about 1.8 GiB |
build |
about 0.5 GiB |
A guest kernel build is the long pole. Budget wall-clock time in tens of minutes on 8 cores. QEMU is comparable on a first build and about a minute for an incremental single-file rebuild.
Cross-building the ORV3 satellite firmware needs arm-none-eabi-gcc. The host
unit tests for the same firmware need only the native compiler, which is why CI
runs the tests without a cross toolchain.
6. Tier E, many racks¶
A rack is eight sleds, one virtual switch and one CnuasLink switch, and the sleds dominate. With the default sled, 4 GiB of guest RAM, 256 MiB of device memory and 4 vCPUs, one rack costs about 38 GiB of RAM, 32 vCPUs and 96 GiB of disk. A deployment costs about 1.6 GiB of RAM and 30 GiB of disk once, for the rack management controller, the shelf daemon and the Rack UI.
| Host | Racks per host | Limited by |
|---|---|---|
| 16 cores, 64 GiB, 500 GiB | 1 | Memory |
| 64 cores, 512 GiB, 4 TiB | 7 | Cores |
| 128 cores, 1 TiB, 8 TiB | 15 | Cores |
The core figures assume four guest vCPUs per host core, which suits sleds that are mostly idle. Set the ratio to one where every sled is expected to be busy at once, and the rack counts fall accordingly. Memory becomes the limit again as soon as sleds are given more RAM or more device memory.
Beyond one host, racks are added by adding machines. Reaching 100 racks
takes about 15 machines of 64 cores and 512 GiB, and 1,000 racks takes
about 143 of them. The calculator on the Rack UI at /calculator works
these figures for a given machine and sled shape, and
cnuas_facility.sizing is the module behind it.
Instances on separate machines are joined at the uplink port of their virtual switches, so a fleet presents one fabric rather than several. See the Demonstration Runbook section 6.2 for the mechanism. The join is a TCP connection carrying one Ethernet frame at a time, so ordinary networking between the machines is enough, and they need to reach each other on the chosen port.
7. What is not required¶
To avoid wasted effort, none of the following is needed by any tier.
| Not required | Why |
|---|---|
| A physical GPU | The GPU is emulated in software |
| CUDA or ROCm | Cnuas implements its own runtime and ISA |
| A physical NIC beyond one for networking | The fabric is emulated |
| RDMA-capable hardware | The verbs provider drives the emulated device |
| An ORV3 shelf, PSU or BBU | Renode runs the satellite firmware |
| A BMC | OpenBMC runs on an emulated AST2600 |
| Root, for tier A | The Soft-GPU is an unprivileged process |
Root is needed for tier B and above only to read /dev/kvm and to load the
guest kernel modules inside the guest, not on the host.
There is no host kernel module and no /dev/cnuasgpuN on the host. The Soft-GPU
is a userspace library. The reasoning is recorded in
CnuasGPU as a Soft-GPU section 1.1.
8. Checking a host¶
The control plane reports what it found rather than what it expected.
For the accelerator specifically, cnuassmi prints the backend in use and the
device memory it mapped. A row of zeroes means the device failed to open, most
often because the requested arena could not be mapped.
KVM availability is worth checking before tier B rather than after a slow boot.
test -r /dev/kvm && echo "KVM usable" || echo "KVM not usable, TCG only"
grep -o 'avx512[a-z]*' /proc/cpuinfo | sort -u | head
9. Related reading¶
| Document | Covers |
|---|---|
| Installation Guide | Getting the software onto a host |
| User Guide | Configuration and day to day operation |
| Developer Guide | Building, APIs, CI |
| CnuasGPU as a Soft-GPU | Host accelerator design |
| CnuasGPU Design | Emulated device design |