A Holochain edgenode fleet

Five nodes built from the nixos-holochain modules, node-01 doubling as the Grafana monitor node, plus a live ISO to install them from. Generated by nix flake init -t github:Sensorica/nixos-holochain#fleet.

Layout

.
├── flake.nix                      # inputs, the five nixosConfigurations, the ISO, the colmena hive
├── hosts/
│   ├── common.nix                 # shared by every host: user, SSH keys, desktop, edgenode service
│   ├── node-01/
│   │   ├── configuration.nix      # monitor node: adds Grafana and Prometheus
│   │   └── hardware-configuration.nix   # placeholder, replace per machine (below)
│   ├── node-02 … 05/              # peer nodes: hostname + hardware only
│   └── live-iso/configuration.nix # KDE Plasma live ISO
└── README.md

First steps

Rename the hosts/node-0* directories and the hosts list in flake.nix to your machines’ names, and update networking.hostName in each configuration.nix and the scrapeTargets list in node-01.
Paste your SSH public key into the operatorKeys list at the top of hosts/common.nix; it goes on the operator account and on root, which Colmena connects as. A fleet deployed with that list empty has no way in over SSH.
Replace each placeholder hardware-configuration.nix (see below).

Evaluate

nix flake check --no-build
nix eval .#nixosConfigurations.node-01.config.system.build.toplevel.drvPath

Hardware configuration

Each host ships a placeholder hardware-configuration.nix so the fleet evaluates before any machine exists. Before deploying to real hardware, generate the real one on that machine and commit it over the placeholder:

sudo nixos-generate-config --show-hardware-config > hosts/node-01/hardware-configuration.nix

Nothing needs keeping from the placeholder: nixos-generate-config --show-hardware-config writes filesystems and kernel modules, never a boot loader, and the GRUB block that serves both firmwares lives in hosts/common.nix.

Firmware assumption

The placeholder targets a machine that may boot legacy BIOS or UEFI, because the fleet this template came from is built on Holoports (legacy BIOS only) and installed from laptops that are usually UEFI. So the disk is GPT with a 1 MiB bios_grub partition and a vfat ESP labelled boot, an ext4 root labelled nixos and a swap partition labelled swap, and GRUB is installed twice:

the UEFI half by NixOS from boot.loader.grub in hosts/common.nix (device = "nodev", efiSupport, efiInstallAsRemovable, ESP mounted at /efi-boot);
the BIOS half by one command in the install runbook, grub-install --target=i386-pc --boot-directory=/mnt/boot /dev/sda.

efiInstallAsRemovable writes EFI/BOOT/BOOTX64.EFI, so firmware that keeps no boot variables still finds it. If your machines are UEFI only you can drop the bios_grub partition and the i386-pc command; if they are BIOS only, the ESP and the EFI half are what you drop. Layout and both commands after holochain/wind-tunnel-runner (base-install.nix, installer.nix); the whole partition, install and grub-install sequence is one command, nix run github:Sensorica/nixos-holochain#holoport-install -- DISK FLAKE#HOST, written up in the upstream docs/deployment.md § “Installing on a Holoport (legacy BIOS)”.

Deploy

# one machine
sudo nixos-rebuild switch --flake .#node-01

# the whole fleet over SSH, in parallel
nix develop            # brings colmena into PATH
colmena apply --impure --on @all
colmena apply --impure --dry-run

--impure is required with Colmena 0.4.0 on Nix 2.25: Colmena wraps the flake as an input named hive, and pure mode refuses to lock it (cannot update unlocked flake input 'hive' in pure mode). Colmena resolves nixos-holochain from this directory’s flake.lock, so --override-input does not reach it; run nix flake update nixos-holochain to deploy modules newer than the locked revision.

Live ISO

nix build .#nixosConfigurations.live-iso.config.system.build.isoImage
sudo dd if=result/iso/*.iso of=/dev/sdX bs=4M status=progress
sync

Monitoring

node-01 serves Grafana on :3000 with the five Holochain dashboards provisioned, “What is this machine running?” as its home page, scraping every node’s node_exporter and the conductor metrics timer. It logs in as admin with the password in /var/lib/secrets/grafana-admin-password, which you create on the node before the first deploy (root-owned, mode 0400; systemd hands it to Grafana); services.holochain-grafana.adminPasswordFile in the option reference gives the commands. The module’s adminPassword default is a lab convenience and lands world-readable in the Nix store, so it is not used here.

Option reference

Every option used here is documented in docs/module-options.md in the module repository.

Keyboard shortcuts

nixos-holochain