Development plan¶
PyrixOS is designed before it is built. Each phase starts only when the one before it is complete, except Phase 7, which needs none of the others, and implementation starts only when its RFC is accepted.
| Phase | Scope | Deliverables | Done when |
|---|---|---|---|
| 0. Architecture RFC | Agree the design of the language, the build sandbox, the store, the garbage collector, profiles and binary caches | RFC 0001 PyrixOS Architecture, published on the documentation site | The RFC has been reviewed in public, its open questions are resolved or deferred to named follow-up RFCs, and its status is Accepted. |
| 1. PyBonsai evaluator | Interpret a restricted Python syntax tree and produce deterministic derivations | Hermetic evaluator, derivation format, Lattice standard library, Lattice library interface with immutable data models, lattice daemon with JSON-RPC and REST over its Unix socket and TLS, Client, evaluator and daemon test suites |
Permitted constructs evaluate correctly. Imports, while, I/O, mutation, recursion, dunder access and unsupported nodes are rejected. Identical derivations hash identically on different machines. A file evaluates to the same derivations through the library, the daemon and the remote API, and callers without a permitted role or credential are refused. |
| 2. Build sandbox | Run builds in isolated, hardened Linux namespaces with no network access | Namespace backend, seccomp filter and resource limits, fixed build environment, sandbox filesystem, build runner, lattice sandbox-config, sandbox test suite, all but the derivation rules published as the standalone Cloche package with its cloche command |
Builds run in user, mount, network, PID, UTS and IPC namespaces with no new privileges. Network access, writes outside /build and the build's own outputs, and creating setuid files fail. The build environment is fixed. Missing outputs fail the build and temporary state is removed. |
| 3. Store and garbage collector | Keep build outputs in an immutable store tracked in SQLite | Store publication, dependency database, parallel builder, fetch_url downloads, garbage collection roots, garbage collector, lattice build --check, lattice plan, lattice verify, store test suite |
Outputs move atomically into the store and become read-only. Writes to the store fail. Collection keeps everything reachable from a root and removes the rest. Rebuilding a reproducible path with --check reports no differences. Users without write access to the store build through the daemon, and only the daemon writes to the store. |
| 4. Profiles and stacks | Select what is active through generations, with one active stack per slot | Profile generations, rollback, generation deletion, stack slots, activation with kernel module checks and undo, deferral to the next boot, lattice activate-boot, activation test suite |
Switching and rolling back are atomic. A slot never holds two stacks. Kernel modules change only while their devices are unused, and a stack in use by a running process is never collected. Through the daemon, a user switches only their own profiles. |
| 5. Binary caches | Fetch prebuilt store paths instead of building them | Cache format, signing, lattice generate-key, SLSA provenance, lattice push, HTTP substitution, lattice substitute, CernVM-FS full copies and shallow paths, lattice materialise, cache test suite |
Unsigned or altered paths are rejected. A path fetched from a cache is identical to one built locally. Shallow paths work as slots, kernel modules are always local, and collection removes only the symlink of a shallow path. Every published path has signed provenance, and a machine that requires provenance rejects a path without it. |
| 6. Container and microVM backends | Run builds in an OCI container runtime or a lightweight virtual machine for stronger isolation and less dependence on the host | OCI backend, microVM backend for Firecracker, Cloud Hypervisor and OpenVMM with the guest kernel and init as store paths and recorded in provenance, lattice check --backend, backend test suite |
The same derivations give identical outputs on every backend. The virtual machine has no network device. An unavailable backend stops the build instead of falling back to a weaker one. |
| 7. Configuration system | Choose system options from a Kconfig-style tree and check with a SAT solver that the choices fit together | RFC 0003 Configuration System, Kconfig subset parser and checker, encoding into clauses, built-in CDCL solver, optional PicoSAT backend, resolution, explanations of conflicts with fixes, defconfig and config.pyb files, trellis menu and trellis old consoles, the other trellis commands, lattice config, published as the standalone Trellis package, test suite |
Constructs outside the subset and files outside the tree are refused. The same tree and choices give the same configuration with every solver backend. A conflict names the choices and rules involved and lists the smallest sets of choices to drop, and nothing is written while the choices conflict. |
| 8. Component releases | Publish PyBonsai, Cloche, Trellis and Lattice as separate packages that other projects can install and use | Version 1.0.0 of pybonsai-lang, cloche and trellis-kconfig, version 0.9.0 of pyrixos-lattice, a GPL-2.0 licence and change log in each repository, continuous integration on Python 3.12 and 3.13, release workflows that publish to PyPI with trusted publishing and build provenance, the integration guide, RELEASING.md |
Each package installs from PyPI into a clean environment and its public interface works there without the PyrixOS repository. |
| 9. Bootstrap | Build the first tools of the distribution from source, starting from a small fixed seed | RFC 0002 Bootstrap, the binary seed fetched by SHA-256, stage builds of a C toolchain, shell and core utilities, a rebuild of the seed from the stage tools, bootstrap test suite | Every package after the seed is built from source in the sandbox. Two builds of the stage tools on different machines give identical store paths. |
| 10. System configuration | Describe a whole machine in PyBonsai and switch to it | RFC 0004 System Configuration, users and groups, file systems, services as systemd units, the boot loader entry for each generation, config.pyb read through the config mapping, read-only bind mount of the store, cgroup limits for builds, lattice switch |
A system description builds to one store path, switches atomically, and rolls back to the previous generation, also from the boot menu. |
| 11. Package set and images | Provide the base packages and installable images | RFC 0005 Package Set and Images, the package set in layers with its standard environment, Linux kernel with a defconfig and fragments, systemd, base user space, Python, networking and an editor as PyBonsai descriptions, a bootable disk image and an installer image, image builds in continuous integration, a public binary cache | The installer image boots in a virtual machine and on one physical x86_64 machine, installs a system, and the installed system updates from the public cache. |
| 12. Architectures, images and boot loaders | Build PyrixOS for RISC-V 64 and ARM64 as well as x86_64, in the image formats users install from, with the boot loaders their machines need | RFC 0006 Architectures, Image Formats and Boot Loaders, the Trellis option ARCH, package set layers that read it, riscv64 and aarch64 builds through emulation and on target hardware, hybrid ISO, network boot, qcow2, VHDX, VDI and WSL images, systemd-boot, GRUB for BIOS and U-Boot board images, a QEMU boot test per architecture and format |
The riscv64 and then the aarch64 test image boot to running in QEMU, every x86_64 store path is unchanged by the work, and each image format passes its boot or content test. |
| 13. PyrixOS 1.0 release | Freeze the formats and publish the first release | RFC 0001 to 0006 Accepted, pyrixos-lattice 1.0.0, signed images and cache keys, release notes, installation guide |
The formats of derivations, the store database, profiles and caches are stable, and 1.x releases read everything 1.0 wrote. |
| 14. AI and HPC stack | Package the software of GPU and high performance computing machines, using the components and versions of azhpc-images as the reference | Recipes built from source for rdma-core, libfabric, PMIx, Open MPI, MVAPICH, UCX, gdrcopy, NCCL with nccl-tests, RCCL, nvbandwidth and mpifileutils; recipes that unpack the CUDA toolkit, DCGM and the NVIDIA fabric manager; the NVIDIA open kernel modules built against the PyrixOS kernel; a way to mark a recipe whose licence forbids redistribution and keep its outputs out of the public cache; a GPU system configuration and image | The open source parts build in the sandbox and are served from the public cache, the recipes with restricted licences build locally from the vendor downloads, and a GPU machine boots the image and passes the NCCL all-reduce test across its GPUs. |
Every phase delivers its features through three interfaces at once: as methods of the Lattice Python class, through lattice daemon on its Unix socket, and through the remote API over TLS, in both JSON-RPC and REST style. The lattice command only calls those methods. Programs other than the command can therefore use each feature as soon as its phase is complete (RFC 0001 sections 6.1 and 6.2).
Phases 1 to 7 each report timing for their main operation. These are evaluation time per syntax tree node, sandbox setup and teardown time, virtual machine boot time, garbage collector traversal time, substitution time, time to first open of a file in a shallow path, and the time the solver takes to resolve a configuration.