CnuasSHMEM, Datasheet¶
| Item | Value |
|---|---|
| Part | CnuasSHMEM |
| Type | Partitioned global address space communication library |
| Models | NVIDIA NVSHMEM |
| Status | Preview, specified but not implemented |
| Repo | PacketFive/CnuasGPU |
Preview datasheet
CnuasSHMEM carries no implementation in the tree. No symmetric heap, no one sided operations and no signalling exist. This datasheet records the specified design.
1. Overview¶
CnuasSHMEM is the intended one sided communication library for CnuasGPU. Where CnuasCCL provides collective operations that every rank enters together, CnuasSHMEM lets one accelerator read and write another accelerator's memory without the remote side taking part, which suits irregular and latency sensitive access patterns.
2. Specified operations¶
| Group | Operations |
|---|---|
| One sided data movement | Put, get |
| Atomics | Remote read modify write |
| Signalling | Set and wait on a remote flag |
3. Transport¶
| Item | Specified value |
|---|---|
| Primary transport | CnuasLink, accelerator to accelerator |
| Address space | Symmetric heap across participating accelerators, not specified in detail |
| Underlying path | The accelerator fabric rings described in Kernel Modules section 5 |
4. Dependencies¶
| Requirement | Status |
|---|---|
| CnuasLink transport | Implemented |
| Remote memory addressing model | Not specified |
| Device side calls from within a kernel | Not implemented, needs CnuasCC |
| Atomic operation support on the fabric | Not implemented |
Note that one sided operations issued from inside a kernel, which is the usual NVSHMEM pattern, need a device compiler. Host initiated operations could land earlier, and that split has not yet been decided.
5. Interim position¶
Applications needing one sided semantics today should use RDMA verbs directly through the verbs provider, which supports remote write, remote read and atomic operations over the network fabric.
6. Revision history¶
| Revision | Notes |
|---|---|
| A | First publication as a preview. Operation list taken from the CnuasGPU design document, section 7.6. |