Lattice package manager¶
Lattice is the package manager at the centre of PyrixOS. It takes a PyBonsai description and carries it all the way to installed software. It evaluates the description, turns it into derivations, builds each derivation in a sandbox, publishes the results to the store, and later removes what is no longer needed. Lattice is written in Python and depends only on the Python standard library, PyBonsai, Cloche, Espalier for its data models, pyrixos-ed25519 for signatures, and the Linux kernel. All of them are pure Python. It needs no container runtime or external build tool. Its daemon is optional and lets several users share one store.
From description to store¶
---
config:
themeVariables:
fontSize: 16px
flowchart:
useMaxWidth: false
sequence:
useMaxWidth: false
---
flowchart TB
a["PyBonsai file"] -->|"parse and interpret"| b["Derivation graph"]
b -->|"already in store?"| c{"Hash known"}
c -->|"yes"| e["Reuse store path"]
c -->|"no"| d["Build in sandbox"]
d --> f["Publish to store"]
f --> g["Record references"]
Lattice works in stages and each one only consumes the output of the stage before it. The evaluator never builds, the sandbox never evaluates, and the store never runs build code. Because every derivation is identified by a hash of all its inputs, Lattice can see that a build has already been done and reuse its result without running it again.
Derivations¶
A derivation is the exact recipe for one build. It names the package, the builder program and its arguments, the environment the builder sees, and the outputs it must produce. Any input that is itself a derivation, such as a source archive or a compiler, is part of the environment.
Lattice serialises each derivation to canonical JSON, with sorted keys, fixed encoding, and every input derivation replaced by its own hash. It then takes the SHA-256 digest of the result. That digest names the store path the build will produce. Changing anything that goes into a build, directly or several dependencies away, therefore produces a different hash and a different path, while identical recipes on different machines produce the same one.
Standard library¶
The standard library is the only way a description reaches anything outside itself, and every function in it is pure.
| Function | Purpose |
|---|---|
derivation |
Creates a derivation from a name, builder, arguments, environment and outputs. |
fetch_url |
Declares a source by https or http URL and expected SHA-256. The download happens at build time, outside the sandbox, and fails if the hash does not match. |
machine |
Describes a whole machine: kernel, packages, stacks, users, groups, file systems, services and files under /etc. It builds into one store path that lattice switch activates and adds to the boot menu (RFC 0004). |
user, group, file_system, service, etc_file |
Validated records that machine reads. Secrets such as password hashes are named by paths outside the store. |
unpack |
Unpacks a tar archive into the store, outside the sandbox, so the bootstrap seed can be used before any program is in the store (RFC 0002). |
closure |
Lists the closure of a derivation, with the registration.json that the first boot of a disk image imports (RFC 0005). |
len, range, sorted, min, max, str, int |
Pure helpers for working with immutable values. |
New functions are added only through an RFC.
The packages of PyrixOS are not part of the standard library. They live in a package set, a directory of layers such as 00-bootstrap.pyb and 10-stdenv.pyb evaluated in the order of their names (RFC 0005). Each layer sees the names of the layers before it, and a description evaluated with --package-set DIR, or by lattice switch with /etc/pyrixos/pkgs, sees them as if they were standard library names.
Using Lattice from Python¶
Lattice is a Python library first, and the lattice command is a thin wrapper around it. Another program, such as a provisioning tool or a test harness, can do everything the command does by creating a Lattice object and calling its methods.
from lattice import Lattice, Settings
lattice = Lattice(Settings(sandbox_backend="namespaces"), on_event=print)
result = lattice.evaluate("hello.pyb")
for name, drv in result.derivations.items():
print(name, drv.store_path)
The library reads no settings from the machine unless asked, keeps no state between calls, returns immutable Espalier data models, raises subclasses of LatticeError, reports progress through the optional on_event callable and the lattice logger, and ships type hints.
Daemon and remote API¶
lattice daemon runs as the lattice user and is then the only process that writes to the store. Other processes reach it through lattice.Client, which has the same methods, results and errors as Lattice.
from lattice import Client
local = Client() # unix:///run/lattice/daemon.sock
remote = Client("https://builder.example:7420", ca_file="ca.pem",
cert_file="me.pem", key_file="me.key")
| Transport | Caller identified by | Roles |
|---|---|---|
Unix socket /run/lattice/daemon.sock |
Kernel peer credentials | Admin for root, the daemon user and lattice-admins; user for lattice-users |
| TLS 1.3 on a configured port | Client certificate or bearer token | As granted in the daemon settings |
Tools in other languages use the same interface directly, as JSON-RPC 2.0 at POST /v1/rpc or as REST at POST /v1/<method>, with the same JSON data models. Users may evaluate, build, check, switch and roll back their own per-user/<name> profiles, and admins may also collect garbage, push to caches, change any profile and switch the whole system. The daemon never opens a file named by a caller and always builds with its own settings. RFC 0001 sections 6.1 to 6.3 define the library, the daemon, the remote API and the settings file.
Measurement¶
Lattice times its own work so that the cost of each stage is visible. It records parse and evaluation time with the number of syntax tree nodes visited, sandbox setup and teardown time for every build, and the time taken and paths visited by each phase of garbage collection. Samples are held in memory and written out after the operation finishes, so timing never adds I/O to the work being timed. The metrics are listed in RFC 0001 §15.