Extending Cnuas¶
This guide is about adding new emulated hardware to a Cnuas rack. It walks the case the architecture was built around, which is a device that sits on the ORV3 RS-485 segment and is read by the rack management controller, and then the second case of a device reached over Ethernet and IP instead.
Read the Developer Guide first for the build system and coding standards.
1. Deciding where a component belongs¶
Cnuas uses three emulators, and picking the wrong one costs far more than the component itself. The choice follows from how the device is attached.
| Attachment | Emulator | Example |
|---|---|---|
| PCI or PCIe inside a compute node | QEMU | CnuasGPU, CnuasNIC |
| RS-485, I2C, PMBus, or a bare microcontroller | Renode | ORV3 PSU and BBU |
| Ethernet and IP | Host daemon or the control plane | CnuasSwitch, CnuasLink |
The reason the ORV3 power shelf runs in Renode rather than QEMU is that the shelf is a multi-drop RS-485 segment with twelve microcontrollers on one pair of wires. QEMU models point to point character devices, not a shared electrical bus where every node hears every frame. Renode models the microcontrollers themselves, which is what a PSU actually is.
This is also why the management controller is OpenBMC rather than a stub. A component is worth emulating only if the real software that would read it can read it unmodified.
2. The chain a new shelf component joins¶
Before adding anything, it is worth knowing what already exists end to end, because a new component plugs into an existing chain rather than building one.
| Layer | Where | Role |
|---|---|---|
| Device model | bmc/firmware/orv3/src |
Registers and plant behaviour, portable C99 |
| Modbus slave core | bmc/firmware/orv3/src/orv3_modbus.c |
Framing, CRC, function dispatch |
| Target glue | bmc/firmware/orv3/target/cortex-m |
Straps, UART, systick, main loop |
| Platform | bmc/renode/platforms |
The microcontroller Renode instantiates |
| Segment | bmc/renode/scripts |
The shared RS-485 wire and the TCP bridge |
| Master | bmc/rmc |
The one Modbus master, sweeping the shelf |
| Publication | bmc/rmc/src/sensors.c |
One D-Bus sensor object per measurement |
| Presentation | OpenBMC | Redfish, busctl, phosphor tooling |
The design rule that makes this work is that a measurement enters as a Modbus register and leaves as a standard OpenBMC sensor object. Nothing downstream knows what Modbus is, so nothing downstream needs changing when a component is added.
3. Adding an RS-485 shelf component¶
The worked example is a new shelf device, such as a fan controller or a shelf sequencer, sitting alongside the existing PSUs and BBUs.
3.1 Choose a unit address¶
ORV3 addresses are one byte, laid out as two type bits, three rack bits and three device bits.
| Type bits | Device | Base address |
|---|---|---|
11 |
PSU | 0xC0 |
01 |
BBU | 0x40 |
The address for a slot is the base with the rack and slot shifted in, which is
what orv3_psu_address(rack, slot) computes as 0xC0 | (rack << 3) | slot. A
new device type takes an unused type encoding. Provide the equivalent address
helper so the mapping is expressed once and tested, rather than being written
out at each call site.
3.2 Write the register map¶
Register maps are generated from the vendored rackmon maps rather than typed by hand, and CI checks that the generated files are in sync. Do not edit the generated headers.
A register descriptor carries its start, length, format, and the conversion between engineering units and the wire encoding.
typedef struct {
uint16_t begin;
uint16_t length;
orv3_reg_format_t format;
uint8_t precision;
double scale;
double shift;
bool is_signed;
const char *name;
} orv3_reg_desc_t;
The float conversion is value = (raw / 2^precision) * scale + shift, which
covers both the PSU fixed-point registers and the BBU temperatures held in
decikelvin. Use the existing helpers rather than open coding a conversion.
| Helper | Purpose |
|---|---|
orv3_encode_value |
Engineering value to raw register |
orv3_decode_value |
Raw register to engineering value |
orv3_find_reg |
Descriptor lookup by offset |
orv3_store_string |
ASCII into a register range, as the FRU strings are held |
3.3 Write the device model¶
Follow the shape of orv3_psu.h. A device model owns its register file, a set
of plant inputs driven by the rack simulation, and its internal state.
typedef struct {
uint16_t regs[MY_DEVICE_REG_SPAN];
orv3_modbus_slave_t slave;
/* Plant inputs, driven by the rack simulation. */
double inlet_temp_c;
bool enabled;
/* Internal plant state. */
double up_time_s;
} my_device_t;
bool my_device_init(my_device_t *dev, uint8_t unit_address, uint8_t slot);
void my_device_tick(my_device_t *dev, double dt_seconds);
void my_device_set_alarm(my_device_t *dev, uint8_t alarm_bit, bool asserted);
Three requirements are not optional.
| Requirement | Reason |
|---|---|
| Portable C99, no MCU or host dependency | The same source builds as firmware and as a host test |
init rejects an address outside its range |
A bad strap is a wiring fault and must not look like a working device |
| Telemetry moves coherently | Values that do not respond to load teach nothing |
The last point is what separates a model from a stub. In the PSU, output power tracks the load the rack places on the shelf, input power follows output power through an efficiency curve, and temperature and fan speed follow dissipation. A device whose registers hold constants would pass a read test and would be worthless in a rack simulation.
3.4 Attach to the Modbus core¶
The Modbus core never assumes a flat array. The device model supplies read and write callbacks, so a model may synthesise a value on read.
static bool my_read_reg(orv3_modbus_slave_t *slave, uint16_t addr, uint16_t *out);
static bool my_write_reg(orv3_modbus_slave_t *slave, uint16_t addr, uint16_t value);
Return false to raise an illegal data address exception. Do not return a
zero for an unmapped register. A master that reads an address the device does
not implement should be told so, because that is what real hardware does and
because a silently zeroed register is the hardest kind of emulation bug to
find.
Supported function codes are read holding registers, write single, write multiple, and read file record. The core already enforces the RTU limits of 125 registers per read and 123 per write, and maintains the diagnostic counters that real ORV3 devices expose for frames received, CRC errors, exceptions and frames ignored.
3.5 Add the personality to the firmware¶
One firmware image serves every node on the shelf. At boot it reads the strap
block and becomes the device the straps select. Extend the dispatch in
bmc/firmware/orv3/target/cortex-m/main.c with the new device type, following
the existing PSU and BBU branches, including the behaviour of parking the node
when the strap is out of range.
The strap block is five words at 0x30000000.
| Offset | Meaning |
|---|---|
0x00 |
Magic, 0x4F525633 |
0x04 |
Device type |
0x08 |
Modbus unit address |
0x0C |
Shelf slot |
0x10 |
Rack load in milliamps |
The first four are straps and are read once at boot, mirroring the address pins and shelf position on real hardware. The fifth is not a strap. It is re-read on every plant tick, so the load can be changed while the shelf runs.
3.6 Add the Renode platform if needed¶
An existing device that is another Cortex-M node on the same wire needs no new
platform. Reuse cnuas-orv3-satellite.repl.
A genuinely different microcontroller needs its own platform file. Addresses in
it must match target/cortex-m/board.h and link.ld, because nothing checks
that they agree and a mismatch presents as a node that boots and never answers.
cpu: CPU.CortexM @ sysbus
flash: Memory.MappedMemory @ sysbus 0x00000000
sram: Memory.MappedMemory @ sysbus 0x20000000
straps: Memory.MappedMemory @ sysbus 0x30000000
rs485: UART.NS16550 @ sysbus 0x40000000
3.7 Put the node on the wire¶
Add the node to bmc/renode/scripts/cnuas-orv3-power-shelf.resc following the
existing entries. The shared wire is an RS-485 segment provided by a plugin in
the tree, because a stock Renode UART hub with a socket terminal does not
reproduce the multi-drop behaviour the shelf depends on.
mach create "fan0"
machine LoadPlatformDescription $plat
sysbus LoadELF $fw
$devType=2; $unitAddr=0x80; $slot=0
runMacro $strap
connector Connect sysbus.rs485 rs485bus
Every node on the segment hears every frame and only the node whose unit address matches answers. This is the property that makes multi-drop work and the reason a new node cannot simply be given its own private link.
3.8 Teach the master to sweep it¶
The rack management controller in bmc/rmc is the one Modbus master on the
segment. Add the new device to its sweep alongside the existing PSU and BBU
handling in src/shelf.c.
The RMC has two ways onto the same wire, which matters for development. In the
rack and on the emulated BMC it opens a serial port, where /dev/ttyS5 is the
UART wired to the RS-485 transceiver. During development it opens the TCP
endpoint the Renode segment exposes on port 3485. The same framing and decoding
code runs either way, and both build on the host, so the protocol is covered by
tests that need no emulator at all.
3.9 Publish the measurements¶
Add the readings to bmc/rmc/src/sensors.c. OpenBMC has exactly one way to
expose a reading, which is an object under /xyz/openbmc_project/sensors
implementing xyz.openbmc_project.Sensor.Value. Publish one such object per
measurement.
Once that is done the new component appears in busctl, in the object mapper,
in phosphor tooling and in Redfish, without any of them being modified and
without any of them knowing that Modbus was involved.
3.10 Test it¶
Write host unit tests in bmc/firmware/orv3/test. These build natively and are
what CI runs, since CI carries no simulator.
make -C bmc/firmware/orv3 test
make -C bmc/firmware/orv3 check-regmap
make -C bmc/firmware/orv3 firmware
renode bmc/renode/scripts/cnuas-orv3-power-shelf.resc
Cover at least the following, because each is a defect that a naive test would miss.
| Case | Why |
|---|---|
| Correct address answers, others stay silent | Multi-drop correctness |
| A bad CRC is counted and not answered | Real masters rely on this |
| An unmapped register raises an exception | Not a silent zero |
| Encode and decode round trip | Scaling errors are invisible otherwise |
| Telemetry responds to a load change | Proves a model rather than a stub |
| An out of range strap parks the node | A wiring fault must not look healthy |
Then update Validation Matrix with the new case counts. The paper cites those numbers, so they move together.
4. Adding a component reached over Ethernet and IP¶
A component that is network attached rather than on the shelf wire takes a different path, because there is no microcontroller to emulate.
4.1 Where the model lives¶
| Kind of component | Where it goes |
|---|---|
| A switch, fabric element or network service | A host daemon, as CnuasSwitch and CnuasLink are |
| Something the BMC must expose | The RMC, published as a D-Bus sensor object |
| Something the control plane must expose | A new adapter in cnuas.adapters |
Host daemons take a management socket carrying newline framed JSON, which is what the control plane speaks to them. Follow the existing switch and fabric daemons rather than inventing a second convention.
4.2 Add a control-plane adapter¶
Create a module in cnuas/src/cnuas/adapters/, following the shape of
switch.py or sled.py. An adapter converts between the component's protocol
and the service layer, and does nothing else.
Add the socket path or endpoint to cnuas.config.Settings and to
Settings.from_env, with an environment variable and a default. Configuration
that is not in Settings is configuration that cannot be overridden in CI.
4.3 Add service methods, then front ends¶
Put the behaviour in cnuas/src/cnuas/service.py. Then add a CLI command and a
REST route that each call the service method.
Do not put behaviour in a route or in a CLI callback. The reason the CLI and the REST API cannot disagree today is that neither contains any logic, and that property is only preserved by keeping it.
Raise CnuasError and its subclasses for failures. A single exception handler
maps them to HTTP responses, so error shape stays uniform without per route
work.
4.4 If it speaks Redfish¶
Cnuas is a Redfish client, not a server. A component that should be managed
over Redfish is exposed by OpenBMC, and Cnuas reads it, as the sled adapter
reads /redfish/v1/Systems/system.
Resist the temptation to add a Redfish server to Cnuas for a new component. A Redfish implementation written for the emulator would be tested against itself and would prove nothing about real rack management software. Publishing a D-Bus sensor object and letting OpenBMC's Redfish front end present it gives a real implementation for free.
4.5 Test it¶
Adapters are testable without the component running, and should be. A test that requires the daemon to be up cannot run in CI.
5. Adding a PCI device to a compute node¶
For completeness, a device inside a compute node is a QEMU device and lives in
qemu/hw/misc alongside cnuasgpu.c, with its guest driver under src/.
The following constraints apply to a QEMU device.
| Point | Consequence of getting it wrong |
|---|---|
| A BAR at or above 4 GiB must be 64-bit prefetchable | Firmware reports it as not mapped and the guest driver cannot map it |
Use DEFINE_PROP_SIZE for a size property |
DEFINE_PROP_UINT64 rejects suffixes such as 8G |
| Validate range and alignment at realise | Better to fail loudly than to realise a device the guest cannot use |
| Rebuild QEMU before testing | A stale binary shows the old behaviour |
Verify a BAR is genuinely mapped rather than merely accepted. With -S the
BARs are never programmed, so firmware must be allowed to run.
(sleep 10; printf 'info pci\nquit\n') | \
./qemu/build/qemu-system-x86_64 -device cnuasgpu,devmem_size=8G -monitor stdio
A correct result shows a 64-bit prefetchable region above the 4 GiB line. A report of not mapped means the BAR cannot describe the size requested.
New device identifiers are not invented. Identifier allocation and the current requests are recorded in Upstreaming. Note that a component with no PCI function, such as a host daemon or a library, needs no device identifier at all.
6. Checklist¶
| Step | Applies to |
|---|---|
| Device model in portable C99 | Shelf components |
| Register map generated, not hand written | Shelf components |
| Read and write callbacks raise exceptions for unmapped registers | Shelf components |
| Personality added to the firmware strap dispatch | Shelf components |
| Node added to the RS-485 segment script | Shelf components |
| RMC sweeps it | Shelf components |
| Published as a D-Bus sensor object | Anything the BMC exposes |
| Adapter, settings, service method, CLI and route | Network components |
| Host tests that need no emulator | Everything |
| Validation matrix updated | Everything |
| Datasheet added or updated | Anything user visible |
7. Related reading¶
| Document | Covers |
|---|---|
| Developer Guide | Build system, APIs, standards, CI |
| OCP Rack Management | BMC, RS-485 and Redfish design |
| Control Plane | Daemon protocols |
| Programmability | CLI and REST reference |
| CnuasGPU Design | A worked QEMU device |
| Validation Matrix | Where evidence is recorded |
| Upstreaming | Identifier allocation |