self
b048a370d2
Update supertest operator guidance to cover both devenv and the pinned repo-local Guix wrapper for scenarios, live inspection, cloud CLIs, and environment variables.
5.6 KiB
Tribes Supertest
tribes-supertest is the cross-repo live scenario test suite for Tribes OS rollout, sync, cluster, and plugin behavior.
Environment
Run commands through the repo development shell. Prefer devenv shell -- <command> when devenv is available; if it is unavailable, or when validating the Guix development environment, use the repo-local Guix wrapper:
devenv shell -- npm run typecheck
./guix/guix-dev -- npm run typecheck
devenv shell -- npm run scenario:single-node-plugin-rollout-rollback
./guix/guix-dev -- npm run scenario:single-node-plugin-rollout-rollback
Do not use raw guix shell for this repo when ./guix/guix-dev is available; the wrapper uses pinned channels from guix/channels.scm. When wrapping shell snippets, use bash -c rather than bash -lc so the Guix shell PATH is not overwritten by login-shell profile setup.
The repo development shell normally provides LEGION_UNLOCK_PASSWORD and LEGION_ACME_EMAIL; do not override them unless a specific run needs it, and call that out.
The development shell also exposes scenario tooling and cloud CLIs including legion, hcloud, ovhcloud, and scw. Use those tools for provider cleanup and inspection.
Scenario Runs
Prefer self-signed certificate mode for supertest scenario runs by default:
SUPERTEST_CERT_MODE=self-signed SUPERTEST_WAIT_ON_FAILURE=1 devenv shell -- npm run scenario:<name>
SUPERTEST_CERT_MODE=self-signed SUPERTEST_WAIT_ON_FAILURE=1 ./guix/guix-dev -- npm run scenario:<name>
Use ACME mode only when the scenario explicitly needs real certificates.
Enable SUPERTEST_WAIT_ON_FAILURE=1 by default for live debugging. On failure the runner captures failure artifacts, then waits five minutes before teardown; press Ctrl-C once, or send SIGINT to the scenario process, to skip the remaining wait and continue cleanup.
Before and after live scenarios, check that provider resources are cleaned up. Use Legion state first, then provider CLIs if needed.
Do not leave cloud nodes running unless the user explicitly wants live inspection.
Supertest scenarios should request exact provider instance types. If the requested instance is unavailable, the runner may wait three minutes and retry once for transient Hetzner capacity errors, then it must stop; do not switch to a different server class or provider unless the user explicitly asks for that run.
Dev Channel Runs
For rapid automatic iteration when explicitly requested, prefer the repo helper:
scripts/test-dev-branch --plugin supertest <scenario>
It prepares the hard-coded guix-tribes dev branch, defaults to self-signed mode, and has prepare, reset, env, ssh, and rpc subcommands. See the README "Dev Branch Helper" section for details.
Use scripts/test-dev-branch reset before a fresh iteration series or whenever guix-tribes master gained fixes that the dev branch may not contain. Dev-channel commits are disposable test proofs; any real product, deployment, package, or scenario fix found during a run must be committed to the owning repo's normal master flow before considering the issue done.
The helper reset path must start from the existing authenticated origin/supertest-dev history and sync the tree to current origin/master while preserving dev channel authorization. Do not rebuild the branch directly from the intro commit or create a merge commit with master as a parent; Guix authentication can reject the resulting dev-signed path.
The helper requires clean source checkouts for the pinned repos. If the active
tribes or plugin checkout has unrelated local work, create a clean worktree
and pass it with --tribes-repo or --plugin-repo instead of stashing or
discarding those changes.
When doing the same workflow manually, keep it isolated to the test run by setting:
SUPERTEST_DEV_CHANNEL_MODE=1SUPERTEST_GUIX_TRIBES_CHANNEL_BRANCHSUPERTEST_GUIX_TRIBES_CHANNEL_COMMITSUPERTEST_GUIX_TRIBES_INTRO_COMMITSUPERTEST_GUIX_TRIBES_INTRO_FINGERPRINTSUPERTEST_GUIX_TRIBES_SIGNER_FINGERPRINTandSUPERTEST_GUIX_TRIBES_SIGNER_LABEL, when different from the intro key
Do not treat a dev-channel pass as a signed guix-tribes master release baseline.
Live Debugging
During supertest runs, prefer the repo helper:
devenv shell -- scripts/supertest-legion node list --json
devenv shell -- scripts/supertest-legion ssh <node>
devenv shell -- scripts/supertest-legion rpc <node> 'Application.started_applications()'
./guix/guix-dev -- scripts/supertest-legion node list --json
./guix/guix-dev -- scripts/supertest-legion ssh <node>
./guix/guix-dev -- scripts/supertest-legion rpc <node> 'Application.started_applications()'
scripts/supertest-legion reads .state/supertest/latest.json and reuses the generated LEGION_STATE_DIR, LEGION_CACHE_DIR, Legion repo, and CLI entrypoint from the current or latest scenario run. Do not enter a second development shell inside the Legion checkout for this; the supertest shell provides the Node tooling and the helper invokes Legion directly.
It is fine to run these devenv shell -- scripts/supertest-legion ... or ./guix/guix-dev -- scripts/supertest-legion ...
inspection commands while a scenario is running or waiting for failure
inspection. The general development-shell caution is about starting unrelated commands in
the same checkout at the same time, not about nested/live-debug helper calls
that are part of an active supertest run.
Other live-debug tools:
legion ssh <node>legion node ssh <node>legion node rpc <node> -- <elixir-expression>- provider CLIs from the development shell:
hcloud,ovhcloud,scw