chore: Bump version to 0.7.0, 1st beta

.claude/skills/nostr-nip-sync.md

@@ -0,0 +1,123 @@
+---
+name: nostr-nip-sync
+description: Check upstream NIP changes in ./docs/nips and assess required updates to our Elixir Nostr server implementation.
+scope: project
+disable-model-invocation: false
+tags:
+  - elixir
+  - nostr
+  - protocol
+  - maintenance
+triggers:
+  - "nostr nips"
+  - "sync with upstream nips"
+  - "check for protocol changes"
+  - "review nip updates"
+---
+
+You are an assistant responsible for keeping this Elixir-based Nostr server aligned with the upstream Nostr Implementation Possibilities (NIPs) specification.
+
+## Goal
+
+When invoked, you will:
+1. Detect upstream changes to the NIPs repository mirrored as a git submodule at `./docs/nips/`.
+2. Understand what those changes mean for a Nostr relay implementation.
+3. Decide whether they require updates to our Elixir server code, configuration, or documentation.
+4. Propose concrete implementation tasks (modules to touch, new tests, migrations, etc.) or explicitly state that no changes are required.
+
+The user may optionally pass additional arguments describing context or constraints. Treat all trailing arguments as a free-form description of extra requirements or focus areas.
+
+## Repository assumptions
+
+- This project is an Elixir Nostr relay / server.
+- The upstream NIPs repository is included as a git submodule at `./docs/nips/`, tracking `nostr-protocol/nips` on GitHub.[web:27]
+- Our implementation aims to conform to all NIPs listed in our project documentation and `CLAUDE.md`, not necessarily every NIP in existence.
+- Our project uses standard Elixir project structure (`mix.exs`, `lib/`, `test/`, etc.).
+
+If any of these assumptions appear false based on the actual repository layout, first correct your mental model and explicitly call that out to the user before proceeding.
+
+## High-level workflow
+
+When this skill is invoked, follow this procedure:
+
+1. **Gather local context**
+   - Read `CLAUDE.md` for an overview of the server’s purpose, supported NIPs, architecture, and key modules.
+   - Inspect `mix.exs` and the `lib/` directory to understand the main supervision tree, key contexts, and modules related to Nostr protocol handling (parsing, persistence, filters, subscriptions, etc.).
+   - Look for any existing documentation about supported NIPs (e.g. `docs/`, `README.md`, `SUPPORT.md`, or `NIPS.md`).
+
+2. **Inspect the NIPs submodule**
+   - Open `./docs/nips/` and identify:
+     - The current git commit (`HEAD`) of the submodule.
+     - The previous commit referenced by the parent repo (if accessible via diff or git history).
+   - If you cannot run `git` commands directly, approximate by:
+     - Listing recently modified NIP files in `./docs/nips/`.
+     - Comparing file contents where possible (old vs new) if the repository history is available locally.
+   - Summarize which NIPs have changed:
+     - New NIPs added.
+     - Existing NIPs significantly modified.
+     - NIPs deprecated, withdrawn, or marked as superseded.
+
+3. **Map NIP changes to implementation impact**
+   For each changed NIP:
+
+   - Identify the NIP’s purpose (e.g. basic protocol flow, new event kinds, tags, message types, relay behaviours).[web:27]
+   - Determine which aspects of our server may be affected:
+     - Event validation and data model (schemas, changesets, database schema and migrations).
+     - Message types and subscription protocol (WebSocket handlers, filters, back-pressure logic).
+     - Authentication, rate limiting, and relay policy configuration.
+     - New or changed tags, fields, or event kinds that must be supported or rejected.
+     - Operational behaviours like deletions, expirations, and command results.
+   - Check our codebase for references to the relevant NIP ID (e.g. `NIP-01`, `NIP-11`, etc.), related constants, or modules named after the feature (e.g. `Deletion`, `Expiration`, `RelayInfo`, `DMs`).
+
+4. **Decide if changes are required**
+   For each NIP change, decide between:
+
+   - **No action required**  
+     - The change is editorial, clarificatory, or fully backward compatible.  
+     - Our current behaviour already matches or exceeds the new requirements.
+
+   - **Implementation update recommended**  
+     - The NIP introduces a new mandatory requirement for compliant relays.
+     - The NIP deprecates or changes behaviour we currently rely on.
+     - The NIP adds new event kinds, tags, fields, or message flows that we intend to support.
+
+   - **Design decision required**  
+     - The NIP is optional or experimental and may not align with our goals.
+     - The change requires non-trivial architectural decisions or policy updates.
+
+   Be explicit about your reasoning, citing concrete NIP sections and relevant code locations when possible.
+
+5. **Produce an actionable report**
+
+   Output a structured report with these sections:
+
+   1. `Summary of upstream NIP changes`  
+      - Bullet list of changed NIPs with short descriptions.
+   2. `Impact on our server`  
+      - For each NIP: whether it is **No action**, **Update recommended**, or **Design decision required**, with a brief justification.
+   3. `Proposed implementation tasks`  
+      - Concrete tasks formatted as a checklist with suggested modules/files, e.g.:
+        - `[ ] Add support for new event kind XYZ from NIP-XX in \`lib/nostr/events/\`.`
+        - `[ ] Extend validation for tag ABC per NIP-YY in \`lib/nostr/validation/\` and tests in \`test/nostr/\`.`
+   4. `Open questions / assumptions`  
+      - Items where you are not confident and need human confirmation.
+
+   When relevant, include short Elixir code skeletons or diff-style snippets to illustrate changes, but keep them focused and idiomatic.
+
+## Behavioural guidelines
+
+- **Be conservative with breaking changes.** If a NIP change might break existing clients or stored data, call that out clearly and recommend a migration strategy.
+- **Prefer incremental steps.** Propose small, well-scoped PR-sized tasks rather than a monolithic refactor.
+- **Respect project conventions.** Match the existing style, naming, and architectural patterns described in `CLAUDE.md` and evident in `lib/` and `test/`.
+- **Keep humans in the loop.** When unsure about how strictly to adhere to a new or optional NIP, surface the tradeoffs and suggest options instead of silently choosing one.
+
+## Invocation examples
+
+The skill should handle invocations like:
+
+- `/nostr-nip-sync`
+- `/nostr-nip-sync check for new NIPs that affect DMs or deletion semantics`
+- `/nostr-nip-sync review changes since last submodule update and propose concrete tasks`
+
+When the user provides extra text after the command, treat it as guidance for what to prioritize in your analysis (e.g. performance, privacy, specific NIPs, or components).
+

.github/workflows/ci.yaml

@@ -69,6 +69,11 @@ jobs:
         with:
           node-version: 24
 
+      - name: Install just
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y just
+
       # Cache deps/ directory — keyed on mix.lock
       - name: Cache Mix deps
         uses: actions/cache@v4
@@ -115,7 +120,8 @@ jobs:
 
       - name: Run Node Sync E2E tests
         if: ${{ matrix.main }}
-        run: mix test.node_sync_e2e
+        run: just e2e node-sync
 
       - name: Run Marmot E2E tests
-        run: mix test.marmot_e2e
+        if: ${{ matrix.main }}
+        run: just e2e marmot

.github/workflows/release.yaml

@@ -68,6 +68,11 @@ jobs:
         with:
           node-version: 24
 
+      - name: Install just
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y just
+
       - name: Cache Mix deps
         uses: actions/cache@v4
         id: deps-cache
@@ -113,10 +118,10 @@ jobs:
         run: mix test --color
 
       - name: Run Node Sync E2E
-        run: mix test.node_sync_e2e
+        run: just e2e node-sync
 
       - name: Run Marmot E2E
-        run: mix test.marmot_e2e
+        run: just e2e marmot
 
       - name: Check for unused locked deps
         run: |

.gitmodules

@@ -1,3 +1,6 @@
 [submodule "marmot-ts"]
 	path = marmot-ts
 	url = https://github.com/marmot-protocol/marmot-ts.git
+[submodule "docs/nips"]
+	path = docs/nips
+	url = https://github.com/nostr-protocol/nips.git

AGENTS.md

@@ -1,11 +1,12 @@
 This is a Nostr server written using Elixir and PostgreSQL.
 
-NOTE: Nostr and NIP specs are available in `~/nostr/` and `~/nips/`.
+NOTE: NIP specs are available in `./docs/nips/`.
 
 ## Project guidelines
 
 - Use `mix precommit` alias when you are done with all changes and fix any pending issues
 - Use the already included and available `:req` (`Req`) library for HTTP requests, **avoid** `:httpoison`, `:tesla`, and `:httpc`.
+- Use semantic prefixes in commit messages (feat:, fix:, docs:, chore:, test:, build:, ci:, bench:, dev:)
 
 <!-- usage-rules-start -->
 

CHANGELOG.md

@@ -0,0 +1,18 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [0.7.0] - 2026-03-20
+
+First beta release!
+
+### Added
+- Configurable WebSocket keepalive support in `Parrhesia.Web.Connection`:
+  - server-initiated `PING` frames
+  - `PONG` timeout handling with connection close on timeout
+- New runtime limit settings:
+  - `:websocket_ping_interval_seconds` (`PARRHESIA_LIMITS_WEBSOCKET_PING_INTERVAL_SECONDS`)
+  - `:websocket_pong_timeout_seconds` (`PARRHESIA_LIMITS_WEBSOCKET_PONG_TIMEOUT_SECONDS`)
+
+### Changed
+- NIP-42 challenge validation now uses constant-time comparison via `Plug.Crypto.secure_compare/2`.

README.md

@@ -4,12 +4,14 @@
 
 Parrhesia is a Nostr relay server written in Elixir/OTP.
 
+**BETA CONDITION – BREAKING CHANGES MAY STILL HAPPEN!**
+
 Supported storage backends:
 
 - PostgreSQL, which is the primary and production-oriented backend
 - in-memory storage, which is useful for tests, local experiments, and benchmarks
 
-**ALPHA CONDITION – BREAKING CHANGES MIGHT HAPPEN!**
+Advanced Nostr features:
 
 - Advanced Querying: Full-text search (NIP-50) and COUNT queries (NIP-45).
 - Secure Messaging: First-class support for Marmot MLS-encrypted groups and NIP-17/44/59 gift-wrapped DMs.
@@ -48,10 +50,23 @@ Current `supported_nips` list:
 - Elixir `~> 1.18`
 - Erlang/OTP 28
 - PostgreSQL (18 used in the dev environment; 16+ recommended)
+- [`just`](https://github.com/casey/just) for the command runner used in this repo
 - Docker or Podman plus Docker Compose support if you want to run the published container image
 
 ---
 
+## Command runner (`just`)
+
+This repo includes a `justfile` that provides a grouped command/subcommand CLI over common mix tasks and scripts.
+
+```bash
+just
+just help bench
+just help e2e
+```
+
+---
+
 ## Run locally
 
 ### 1) Prepare the database
@@ -97,9 +112,9 @@ ws://localhost:4413/relay
 Primary test entrypoints:
 
 - `mix test` for the ExUnit suite
-- `mix test.marmot_e2e` for the Marmot client end-to-end suite
-- `mix test.node_sync_e2e` for the two-node relay sync end-to-end suite
-- `mix test.node_sync_docker_e2e` for the release-image Docker two-node relay sync suite
+- `just e2e marmot` for the Marmot client end-to-end suite
+- `just e2e node-sync` for the two-node relay sync end-to-end suite
+- `just e2e node-sync-docker` for the release-image Docker two-node relay sync suite
 
 The node-sync harnesses are driven by:
 
@@ -108,7 +123,7 @@ The node-sync harnesses are driven by:
 - [`scripts/node_sync_e2e.exs`](./scripts/node_sync_e2e.exs)
 - [`compose.node-sync-e2e.yaml`](./compose.node-sync-e2e.yaml)
 
-`mix test.node_sync_e2e` runs two real Parrhesia nodes against separate PostgreSQL databases, verifies catch-up and live sync, restarts one node, and verifies persisted resume behavior. `mix test.node_sync_docker_e2e` runs the same scenario against the release Docker image.
+`just e2e node-sync` runs two real Parrhesia nodes against separate PostgreSQL databases, verifies catch-up and live sync, restarts one node, and verifies persisted resume behavior. `just e2e node-sync-docker` runs the same scenario against the release Docker image.
 
 GitHub CI currently runs the non-Docker node-sync e2e on the main Linux matrix job. The Docker node-sync e2e remains an explicit/manual check because it depends on release-image build/runtime fidelity and a working Docker host.
 
@@ -124,6 +139,8 @@ The intended in-process surface is `Parrhesia.API.*`, especially:
 - `Parrhesia.API.Admin` for management operations
 - `Parrhesia.API.Identity`, `Parrhesia.API.ACL`, and `Parrhesia.API.Sync` for relay identity, protected sync ACLs, and outbound relay sync
 
+For host-managed HTTP/WebSocket ingress mounting, use `Parrhesia.Plug`.
+
 Start with:
 
 - [`docs/LOCAL_API.md`](./docs/LOCAL_API.md) for the embedding model and a minimal host setup
@@ -131,17 +148,30 @@ Start with:
 
 Important caveats for host applications:
 
-- Parrhesia is still alpha; expect some public API and config churn.
+- Parrhesia is beta software; expect some API and config churn as the runtime stabilizes.
 - Parrhesia currently assumes a single runtime per BEAM node and uses globally registered process names.
 - The defaults in this repo's `config/*.exs` are not imported automatically when Parrhesia is used as a dependency. A host app must set `config :parrhesia, ...` explicitly.
 - The host app is responsible for migrating Parrhesia's schema, for example with `Parrhesia.Release.migrate()` or `mix ecto.migrate -r Parrhesia.Repo`.
 
-If you only want the in-process API and not the HTTP/WebSocket edge, configure:
+### Official embedding boundary
+
+For embedded use, the stable boundaries are:
+
+- `Parrhesia.API.*` for in-process publish/query/admin/sync operations
+- `Parrhesia.Plug` for host-managed HTTP/WebSocket ingress mounting
+
+If your host app owns the public HTTPS endpoint, keep this as the baseline runtime config:
 
 ```elixir
 config :parrhesia, :listeners, %{}
 ```
 
+Notes:
+
+- `listeners: %{}` disables Parrhesia-managed HTTP/WebSocket ingress (`/relay`, `/management`, `/metrics`, etc.).
+- Mount `Parrhesia.Plug` in your host endpoint/router when you still want Parrhesia ingress under the host's single HTTPS surface.
+- `Parrhesia.Web.*` modules remain internal runtime wiring. Use `Parrhesia.Plug` as the documented mount API.
+
 The config reference below still applies when embedded. That is the primary place to document basic setup and runtime configuration changes.
 
 ---
@@ -353,6 +383,8 @@ Parrhesia treats NIP-43 invite requests as synthetic relay output, not stored cl
 | `:max_event_ingest_per_window` | `PARRHESIA_LIMITS_MAX_EVENT_INGEST_PER_WINDOW` | `120` |
 | `:event_ingest_window_seconds` | `PARRHESIA_LIMITS_EVENT_INGEST_WINDOW_SECONDS` | `1` |
 | `:auth_max_age_seconds` | `PARRHESIA_LIMITS_AUTH_MAX_AGE_SECONDS` | `600` |
+| `:websocket_ping_interval_seconds` | `PARRHESIA_LIMITS_WEBSOCKET_PING_INTERVAL_SECONDS` | `30` |
+| `:websocket_pong_timeout_seconds` | `PARRHESIA_LIMITS_WEBSOCKET_PONG_TIMEOUT_SECONDS` | `10` |
 | `:max_outbound_queue` | `PARRHESIA_LIMITS_MAX_OUTBOUND_QUEUE` | `256` |
 | `:outbound_drain_batch_size` | `PARRHESIA_LIMITS_OUTBOUND_DRAIN_BATCH_SIZE` | `64` |
 | `:outbound_overflow_strategy` | `PARRHESIA_LIMITS_OUTBOUND_OVERFLOW_STRATEGY` | `:close` |
@@ -538,33 +570,82 @@ Notes:
 
 ## Benchmark
 
-The benchmark compares two Parrhesia profiles, one backed by PostgreSQL and one backed by the in-memory adapter, against [`strfry`](https://github.com/hoytech/strfry) and [`nostr-rs-relay`](https://sr.ht/~gheartsfield/nostr-rs-relay/) using [`nostr-bench`](https://github.com/rnostr/nostr-bench). Benchmark runs also lift Parrhesia's relay-side limits by default so the benchmark client, not server guardrails, is the main bottleneck.
+The benchmark compares two Parrhesia profiles, one backed by PostgreSQL and one backed by the in-memory adapter, against [`strfry`](https://github.com/hoytech/strfry) and [`nostr-rs-relay`](https://sr.ht/~gheartsfield/nostr-rs-relay/) using [`nostr-bench`](https://github.com/rnostr/nostr-bench). The cloud benchmark target set also includes [`nostream`](https://github.com/Cameri/nostream) and [`Haven`](https://github.com/bitvora/haven). Benchmark runs also lift Parrhesia's relay-side limits by default so the benchmark client, not server guardrails, is the main bottleneck.
 
-`mix bench` is a sequential mixed-workload benchmark, not an isolated per-endpoint microbenchmark. Each relay instance runs `connect`, then `echo`, then `event`, then `req` against the same live process, so later phases measure against state and load created by earlier phases.
+`just bench compare` is a sequential mixed-workload benchmark, not an isolated per-endpoint microbenchmark. Each relay instance runs `connect`, then `echo`, then `event`, then `req` against the same live process, so later phases measure against state and load created by earlier phases.
 
 Run it with:
 
 ```bash
-mix bench
+just bench compare
 ```
 
-Current comparison results from [BENCHMARK.md](./BENCHMARK.md):
+### Cloud benchmark (Hetzner Cloud)
 
-| metric | parrhesia-pg | parrhesia-mem | nostr-rs-relay | mem/pg | nostr-rs/pg |
-| --- | ---: | ---: | ---: | ---: | ---: |
-| connect avg latency (ms) ↓ | 9.33 | 7.67 | 7.00 | **0.82x** | **0.75x** |
-| connect max latency (ms) ↓ | 12.33 | 9.67 | 10.33 | **0.78x** | **0.84x** |
-| echo throughput (TPS) ↑ | 64030.33 | 93656.33 | 140767.00 | **1.46x** | **2.20x** |
-| echo throughput (MiB/s) ↑ | 35.07 | 51.27 | 77.07 | **1.46x** | **2.20x** |
-| event throughput (TPS) ↑ | 5015.33 | 1505.33 | 2293.67 | 0.30x | 0.46x |
-| event throughput (MiB/s) ↑ | 3.40 | 1.00 | 1.50 | 0.29x | 0.44x |
-| req throughput (TPS) ↑ | 6416.33 | 14566.67 | 3035.67 | **2.27x** | 0.47x |
-| req throughput (MiB/s) ↑ | 42.43 | 94.23 | 19.23 | **2.22x** | 0.45x |
+For distributed runs (one server node + multiple client nodes), use:
+
+```bash
+just bench cloud
+# or: ./scripts/run_bench_cloud.sh
+```
+
+or invoke the orchestrator directly:
+
+```bash
+node scripts/cloud_bench_orchestrate.mjs
+```
+
+Prerequisites:
+
+- [`hcloud`](https://github.com/hetznercloud/cli) CLI installed
+- Hetzner Cloud token exported as `HCLOUD_TOKEN`
+- local `docker`, `git`, `ssh`, and `scp` available
+
+Example:
+
+```bash
+export HCLOUD_TOKEN=...
+just bench cloud-quick
+# or: ./scripts/run_bench_cloud.sh --quick
+```
+
+Outputs:
+
+- raw client logs per run: `bench/cloud_artifacts/<run_id>/...`
+- JSONL history entries (local + cloud): `bench/history.jsonl`
+
+Useful history/render commands:
+
+```bash
+# List available machines and runs in history
+just bench list
+
+# Regenerate chart + README table for a machine
+just bench update <machine_id>
+
+# Regenerate from all machines
+just bench update all
+```
+
+Current comparison results:
+
+| metric | parrhesia-pg | parrhesia-mem | strfry | nostr-rs-relay | mem/pg | strfry/pg | nostr-rs-relay/pg |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| connect avg latency (ms) ↓ | 34.67 | 43.33 | 2.67 | 2.67 | 1.25x | **0.08x** | **0.08x** |
+| connect max latency (ms) ↓ | 61.67 | 74.67 | 4.67 | 4.00 | 1.21x | **0.08x** | **0.06x** |
+| echo throughput (TPS) ↑ | 72441.00 | 62704.67 | 61189.33 | 152654.33 | 0.87x | 0.84x | **2.11x** |
+| echo throughput (MiB/s) ↑ | 39.67 | 34.30 | 34.20 | 83.63 | 0.86x | 0.86x | **2.11x** |
+| event throughput (TPS) ↑ | 1897.33 | 1370.00 | 3426.67 | 772.67 | 0.72x | **1.81x** | 0.41x |
+| event throughput (MiB/s) ↑ | 1.23 | 0.87 | 2.20 | 0.50 | 0.70x | **1.78x** | 0.41x |
+| req throughput (TPS) ↑ | 13.33 | 47.00 | 1811.33 | 878.33 | **3.52x** | **135.85x** | **65.88x** |
+| req throughput (MiB/s) ↑ | 0.03 | 0.17 | 11.77 | 2.40 | **5.00x** | **353.00x** | **72.00x** |
 
 Higher is better for `↑` metrics. Lower is better for `↓` metrics.
 
 (Results from a Linux container on a 6-core Intel i5-8400T with NVMe drive, PostgreSQL 18)
 
+![Benchmark chart](./bench/chart.svg)
+
 ---
 
 ## Development quality checks
@@ -578,11 +659,13 @@ mix precommit
 Additional external CLI end-to-end checks with `nak`:
 
 ```bash
-mix test.nak_e2e
+just e2e nak
 ```
 
 For Marmot client end-to-end checks (TypeScript/Node suite using `marmot-ts`, included in `precommit`):
 
 ```bash
-mix test.marmot_e2e
+just e2e marmot
+```
+
 ```

bench/chart.gnuplot

@@ -7,7 +7,7 @@
 # fragment generated by the data-prep step that defines the actual plot
 # directives (handling variable server columns).
 
-set terminal svg enhanced size 1200,900 font "sans,11"
+set terminal svg enhanced size 1200,900 font "sans-serif,11" background "#f3f4f6"
 set output output_file
 
 set style data linespoints

bench/history.jsonl

@@ -1 +1,5 @@
-{"timestamp":"2026-03-18T20:13:21Z","machine_id":"squirrel","git_tag":"v0.5.0","git_commit":"970cee2","runs":3,"servers":{"parrhesia-pg":{"connect_avg_ms":9.333333333333334,"connect_max_ms":12.333333333333334,"echo_tps":64030.333333333336,"echo_mibs":35.06666666666666,"event_tps":5015.333333333333,"event_mibs":3.4,"req_tps":6416.333333333333,"req_mibs":42.43333333333334},"parrhesia-memory":{"connect_avg_ms":7.666666666666667,"connect_max_ms":9.666666666666666,"echo_tps":93656.33333333333,"echo_mibs":51.26666666666667,"event_tps":1505.3333333333333,"event_mibs":1,"req_tps":14566.666666666666,"req_mibs":94.23333333333335},"nostr-rs-relay":{"connect_avg_ms":7,"connect_max_ms":10.333333333333334,"echo_tps":140767,"echo_mibs":77.06666666666666,"event_tps":2293.6666666666665,"event_mibs":1.5,"req_tps":3035.6666666666665,"req_mibs":19.23333333333333}}}
+{"schema_version":2,"timestamp":"2026-03-18T21:35:03Z","machine_id":"agent","git_tag":"v0.6.0","git_commit":"7b337d9","runs":3,"versions":{"parrhesia":"0.6.0","strfry":"strfry 1.0.4 (nixpkgs)","nostr-rs-relay":"nostr-rs-relay 0.9.0","nostr-bench":"nostr-bench 0.4.0"},"servers":{"parrhesia-pg":{"connect_avg_ms":26.666666666666668,"connect_max_ms":45.333333333333336,"echo_tps":68100.33333333333,"echo_mibs":37.233333333333334,"event_tps":1647.3333333333333,"event_mibs":1.0666666666666667,"req_tps":3576.6666666666665,"req_mibs":18.833333333333332},"parrhesia-memory":{"connect_avg_ms":14.666666666666666,"connect_max_ms":24.333333333333332,"echo_tps":55978,"echo_mibs":30.633333333333336,"event_tps":882,"event_mibs":0.5666666666666668,"req_tps":6888,"req_mibs":36.06666666666666},"strfry":{"connect_avg_ms":3,"connect_max_ms":4.666666666666667,"echo_tps":67718.33333333333,"echo_mibs":37.86666666666667,"event_tps":3548.3333333333335,"event_mibs":2.3,"req_tps":1808,"req_mibs":11.699999999999998},"nostr-rs-relay":{"connect_avg_ms":2,"connect_max_ms":3.3333333333333335,"echo_tps":166178,"echo_mibs":91.03333333333335,"event_tps":787,"event_mibs":0.5,"req_tps":860.6666666666666,"req_mibs":2.4}},"run_id":"local-2026-03-18T21:35:03Z-agent-7b337d9","source":{"kind":"local","git_tag":"v0.6.0","git_commit":"7b337d9"},"infra":{"provider":"local"},"bench":{"runs":3,"targets":["parrhesia-pg","parrhesia-memory","strfry","nostr-rs-relay"]}}
+{"schema_version":2,"timestamp":"2026-03-18T22:14:37Z","machine_id":"agent","git_tag":"v0.2.0","git_commit":"b20dbf6","runs":3,"versions":{"parrhesia":"0.2.0","strfry":"strfry 1.0.4 (nixpkgs)","nostr-rs-relay":"nostr-rs-relay 0.9.0","nostr-bench":"nostr-bench 0.4.0"},"servers":{"parrhesia-pg":{"connect_avg_ms":14.666666666666666,"connect_max_ms":25.666666666666668,"echo_tps":77133,"echo_mibs":42.233333333333334,"event_tps":1602.6666666666667,"event_mibs":1.0666666666666667,"req_tps":2418,"req_mibs":12.5},"parrhesia-memory":{"connect_avg_ms":9,"connect_max_ms":16,"echo_tps":64218.333333333336,"echo_mibs":35.166666666666664,"event_tps":1578.3333333333333,"event_mibs":1,"req_tps":2431.3333333333335,"req_mibs":12.633333333333333},"strfry":{"connect_avg_ms":3.3333333333333335,"connect_max_ms":6,"echo_tps":63682.666666666664,"echo_mibs":35.6,"event_tps":3477.3333333333335,"event_mibs":2.2333333333333334,"req_tps":1804,"req_mibs":11.733333333333334},"nostr-rs-relay":{"connect_avg_ms":2.6666666666666665,"connect_max_ms":4.333333333333333,"echo_tps":160009,"echo_mibs":87.63333333333333,"event_tps":762,"event_mibs":0.4666666666666666,"req_tps":831,"req_mibs":2.2333333333333334}},"run_id":"local-2026-03-18T22:14:37Z-agent-b20dbf6","source":{"kind":"local","git_tag":"v0.2.0","git_commit":"b20dbf6"},"infra":{"provider":"local"},"bench":{"runs":3,"targets":["parrhesia-pg","parrhesia-memory","strfry","nostr-rs-relay"]}}
+{"schema_version":2,"timestamp":"2026-03-18T22:22:12Z","machine_id":"agent","git_tag":"v0.3.0","git_commit":"8c8d5a8","runs":3,"versions":{"parrhesia":"0.3.0","strfry":"strfry 1.0.4 (nixpkgs)","nostr-rs-relay":"nostr-rs-relay 0.9.0","nostr-bench":"nostr-bench 0.4.0"},"servers":{"parrhesia-pg":{"connect_avg_ms":13,"connect_max_ms":21.666666666666668,"echo_tps":70703.33333333333,"echo_mibs":38.7,"event_tps":1970.6666666666667,"event_mibs":1.3,"req_tps":3614,"req_mibs":20.966666666666665},"parrhesia-memory":{"connect_avg_ms":13,"connect_max_ms":22.333333333333332,"echo_tps":60452.333333333336,"echo_mibs":33.1,"event_tps":1952.6666666666667,"event_mibs":1.3,"req_tps":3616,"req_mibs":20.766666666666666},"strfry":{"connect_avg_ms":3.6666666666666665,"connect_max_ms":6,"echo_tps":63128.666666666664,"echo_mibs":35.300000000000004,"event_tps":3442,"event_mibs":2.2333333333333334,"req_tps":1804,"req_mibs":11.699999999999998},"nostr-rs-relay":{"connect_avg_ms":2,"connect_max_ms":3.3333333333333335,"echo_tps":164995.33333333334,"echo_mibs":90.36666666666667,"event_tps":761.6666666666666,"event_mibs":0.5,"req_tps":846.3333333333334,"req_mibs":2.333333333333333}},"run_id":"local-2026-03-18T22:22:12Z-agent-8c8d5a8","source":{"kind":"local","git_tag":"v0.3.0","git_commit":"8c8d5a8"},"infra":{"provider":"local"},"bench":{"runs":3,"targets":["parrhesia-pg","parrhesia-memory","strfry","nostr-rs-relay"]}}
+{"schema_version":2,"timestamp":"2026-03-18T22:30:08Z","machine_id":"agent","git_tag":"v0.4.0","git_commit":"b86b5db","runs":3,"versions":{"parrhesia":"0.4.0","strfry":"strfry 1.0.4 (nixpkgs)","nostr-rs-relay":"nostr-rs-relay 0.9.0","nostr-bench":"nostr-bench 0.4.0"},"servers":{"parrhesia-pg":{"connect_avg_ms":11.333333333333334,"connect_max_ms":20.666666666666668,"echo_tps":69139.33333333333,"echo_mibs":37.833333333333336,"event_tps":1938.6666666666667,"event_mibs":1.3,"req_tps":4619.666666666667,"req_mibs":26.266666666666666},"parrhesia-memory":{"connect_avg_ms":10,"connect_max_ms":17.333333333333332,"echo_tps":62715.333333333336,"echo_mibs":34.333333333333336,"event_tps":1573,"event_mibs":1.0333333333333334,"req_tps":4768,"req_mibs":23.733333333333334},"strfry":{"connect_avg_ms":3.3333333333333335,"connect_max_ms":6,"echo_tps":60956.666666666664,"echo_mibs":34.06666666666667,"event_tps":3380.6666666666665,"event_mibs":2.2,"req_tps":1820.3333333333333,"req_mibs":11.800000000000002},"nostr-rs-relay":{"connect_avg_ms":2.6666666666666665,"connect_max_ms":4.333333333333333,"echo_tps":161165.33333333334,"echo_mibs":88.26666666666665,"event_tps":768,"event_mibs":0.5,"req_tps":847.3333333333334,"req_mibs":2.3000000000000003}},"run_id":"local-2026-03-18T22:30:08Z-agent-b86b5db","source":{"kind":"local","git_tag":"v0.4.0","git_commit":"b86b5db"},"infra":{"provider":"local"},"bench":{"runs":3,"targets":["parrhesia-pg","parrhesia-memory","strfry","nostr-rs-relay"]}}
+{"schema_version":2,"timestamp":"2026-03-18T22:36:37Z","machine_id":"agent","git_tag":"v0.5.0","git_commit":"e557eba","runs":3,"versions":{"parrhesia":"0.5.0","strfry":"strfry 1.0.4 (nixpkgs)","nostr-rs-relay":"nostr-rs-relay 0.9.0","nostr-bench":"nostr-bench 0.4.0"},"servers":{"parrhesia-pg":{"connect_avg_ms":34.666666666666664,"connect_max_ms":61.666666666666664,"echo_tps":72441,"echo_mibs":39.666666666666664,"event_tps":1897.3333333333333,"event_mibs":1.2333333333333334,"req_tps":13.333333333333334,"req_mibs":0.03333333333333333},"parrhesia-memory":{"connect_avg_ms":43.333333333333336,"connect_max_ms":74.66666666666667,"echo_tps":62704.666666666664,"echo_mibs":34.300000000000004,"event_tps":1370,"event_mibs":0.8666666666666667,"req_tps":47,"req_mibs":0.16666666666666666},"strfry":{"connect_avg_ms":2.6666666666666665,"connect_max_ms":4.666666666666667,"echo_tps":61189.333333333336,"echo_mibs":34.2,"event_tps":3426.6666666666665,"event_mibs":2.2,"req_tps":1811.3333333333333,"req_mibs":11.766666666666666},"nostr-rs-relay":{"connect_avg_ms":2.6666666666666665,"connect_max_ms":4,"echo_tps":152654.33333333334,"echo_mibs":83.63333333333333,"event_tps":772.6666666666666,"event_mibs":0.5,"req_tps":878.3333333333334,"req_mibs":2.4}},"run_id":"local-2026-03-18T22:36:37Z-agent-e557eba","source":{"kind":"local","git_tag":"v0.5.0","git_commit":"e557eba"},"infra":{"provider":"local"},"bench":{"runs":3,"targets":["parrhesia-pg","parrhesia-memory","strfry","nostr-rs-relay"]}}

config/config.exs

@@ -57,6 +57,8 @@ config :parrhesia,
     max_event_ingest_per_window: 120,
     event_ingest_window_seconds: 1,
     auth_max_age_seconds: 600,
+    websocket_ping_interval_seconds: 30,
+    websocket_pong_timeout_seconds: 10,
     max_outbound_queue: 256,
     outbound_drain_batch_size: 64,
     outbound_overflow_strategy: :close,

config/runtime.exs

@@ -277,6 +277,16 @@ if config_env() == :prod do
         "PARRHESIA_LIMITS_AUTH_MAX_AGE_SECONDS",
         Keyword.get(limits_defaults, :auth_max_age_seconds, 600)
       ),
+    websocket_ping_interval_seconds:
+      int_env.(
+        "PARRHESIA_LIMITS_WEBSOCKET_PING_INTERVAL_SECONDS",
+        Keyword.get(limits_defaults, :websocket_ping_interval_seconds, 30)
+      ),
+    websocket_pong_timeout_seconds:
+      int_env.(
+        "PARRHESIA_LIMITS_WEBSOCKET_PONG_TIMEOUT_SECONDS",
+        Keyword.get(limits_defaults, :websocket_pong_timeout_seconds, 10)
+      ),
     max_outbound_queue:
       int_env.(
         "PARRHESIA_LIMITS_MAX_OUTBOUND_QUEUE",

default.nix

@@ -10,7 +10,7 @@
   vips,
 }: let
   pname = "parrhesia";
-  version = "0.6.0";
+  version = "0.7.0";
 
   beamPackages = beam.packages.erlang_28.extend (
     final: _prev: {

devenv.nix

@@ -78,6 +78,7 @@ in {
     with pkgs;
       [
         just
+        # Mix NIFs
         gcc
         git
         gnumake
@@ -85,6 +86,8 @@ in {
         automake
         libtool
         pkg-config
+        # for tests
+        openssl
         # Nix code formatter
         alejandra
         # i18n
@@ -97,14 +100,18 @@ in {
         mermaid-cli
         # Nostr CLI client
         nak
+        websocat
         # Nostr relay benchmark client
         nostr-bench
         # Nostr reference servers
         nostr-rs-relay
         # Benchmark graph
         gnuplot
+        # Cloud benchmarks
+        hcloud
       ]
       ++ lib.optionals pkgs.stdenv.hostPlatform.isx86_64 [
+        # Nostr reference servers
         strfry
       ];
 

docs/LOCAL_API.md

@@ -3,8 +3,8 @@
 Parrhesia can run as a normal standalone relay application, but it also exposes a stable
 in-process API for Elixir callers that want to embed the relay inside a larger OTP system.
 
-This document describes that embedding surface. The runtime is still alpha, so treat the API
-as usable but not yet frozen.
+This document describes that embedding surface. The runtime is now beta, so treat the API
+as usable with minor churn possible while it stabilizes.
 
 ## What embedding means today
 
@@ -14,6 +14,7 @@ Embedding currently means:
 - the host app provides `config :parrhesia, ...` explicitly
 - the host app migrates the Parrhesia database schema
 - callers interact with the relay through `Parrhesia.API.*`
+- host-managed HTTP/WebSocket ingress is mounted through `Parrhesia.Plug`
 
 Current operational assumptions:
 
@@ -63,9 +64,14 @@ config :parrhesia, ecto_repos: [Parrhesia.Repo]
 
 Notes:
 
-- Set `listeners: %{}` if you only want the in-process API and no HTTP/WebSocket ingress.
-- If you do want ingress, copy the listener shape from the config reference in
-  [README.md](../README.md).
+- `listeners: %{}` is the official embedding pattern when your host app owns the HTTPS edge.
+- `listeners: %{}` disables Parrhesia-managed ingress (`/relay`, `/management`, `/metrics`, etc.).
+- Mount `Parrhesia.Plug` from the host app when you still want Parrhesia ingress behind that same
+  HTTPS edge.
+- `Parrhesia.Web.*` modules are internal runtime wiring. Treat `Parrhesia.Plug` as the stable
+  mount API.
+- If you prefer Parrhesia-managed ingress instead, copy the listener shape from the config
+  reference in [README.md](../README.md).
 - Production runtime overrides still use the `PARRHESIA_*` environment variables described in
   [README.md](../README.md).
 
@@ -77,6 +83,27 @@ Parrhesia.Release.migrate()
 
 In development, `mix ecto.migrate -r Parrhesia.Repo` works too.
 
+## Mounting `Parrhesia.Plug` from a host app
+
+When `listeners: %{}` is set, you can still expose Parrhesia ingress by mounting `Parrhesia.Plug`
+in your host endpoint/router and passing an explicit listener config:
+
+```elixir
+forward "/nostr", Parrhesia.Plug,
+  listener: %{
+    id: :public,
+    transport: %{scheme: :https, tls: %{mode: :proxy_terminated}},
+    proxy: %{trusted_cidrs: ["10.0.0.0/8"], honor_x_forwarded_for: true},
+    features: %{
+      nostr: %{enabled: true},
+      admin: %{enabled: true},
+      metrics: %{enabled: true, access: %{private_networks_only: true}}
+    }
+  }
+```
+
+Use the same listener schema documented in [README.md](../README.md).
+
 ## Starting the runtime
 
 In the common case, letting OTP start the `:parrhesia` application is enough.

docs/slop/ALPHA_REVIEW.md

@@ -0,0 +1,351 @@
+# Parrhesia Alpha Code Review
+
+**Reviewer:** Claude Opus 4.6 (automated review)
+**Date:** 2026-03-20
+**Version:** 0.6.0
+**Scope:** Full codebase review across 8 dimensions for alpha-to-beta promotion decision
+
+---
+
+## 1. Executive Summary
+
+Parrhesia is a well-architected Nostr relay with mature OTP supervision design, comprehensive telemetry, and solid protocol implementation covering 15+ NIPs. The codebase demonstrates strong Elixir idioms — heavy ETS usage for hot paths, process monitoring for cleanup, and async-first patterns. The test suite (58 files, ~8K LOC) covers critical protocol paths including signature verification, malformed input rejection, and database integration.
+
+**Most critical gaps:** No WebSocket-level ping/pong keepalives, no constant-time comparison for NIP-42 challenge validation, and the `effective_filter_limit` function can return `nil` when called outside the standard API path (though the default config sets `max_filter_limit: 500`). Property-based testing is severely underutilised despite `stream_data` being a dependency.
+
+**Recommendation:** **Promote with conditions** — the codebase is production-quality for beta with two items to address first.
+
+---
+
+## 2. Dimension-by-Dimension Findings
+
+### 2.1 Elixir Code Quality
+
+**Rating: ✅ Good**
+
+**Supervision tree:** Single-rooted `Parrhesia.Runtime` supervisor with `:one_for_one` strategy. 12 child supervisors/workers, each with appropriate isolation. Storage, subscriptions, auth, sync, policy, tasks, and web endpoint each have their own supervisor subtree. Restart strategies are correct — no `rest_for_one` or `one_for_all` where unnecessary.
+
+**GenServer usage:** Idiomatic throughout. State-heavy GenServers (Subscriptions.Index, Auth.Challenges, Negentropy.Sessions) properly monitor owner processes and clean up via `:DOWN` handlers. Stateless dispatchers (Fanout.Dispatcher) use `cast` appropriately. No unnecessary `call` serialisation found.
+
+**ETS usage:** 9+ ETS tables with appropriate access patterns:
+- Config cache: `:public`, `read_concurrency: true` — correct for hot-path reads
+- Rate limiters: `write_concurrency: true` — correct for high-throughput counters
+- Subscription indices: `:protected`, `read_concurrency: true` — correct (only Index GenServer writes)
+
+**Error handling:** Consistent `{:ok, _}` / `{:error, _}` tuples throughout the API layer. No bare `throw` found. Connection handler wraps external calls in try/catch to prevent cascade failures. Rate limiter fallback returns `:ok` on service unavailability (availability over correctness — documented trade-off).
+
+**Pattern matching:** Exhaustive in message handling. Protocol decoder has explicit clauses for each message type with a catch-all error clause. Event validator chains `with` clauses with explicit error atoms.
+
+**Potential bottlenecks:**
+- `Subscriptions.Index` is a single GenServer handling all subscription mutations. Reads go through ETS (fast), but `upsert`/`remove` operations serialise through the GenServer. At very high subscription churn (thousands of REQ/CLOSE per second), this could become a bottleneck. Acceptable for beta.
+- `Negentropy.Sessions` holds all session state in process memory. Capped at 10K sessions with idle sweep — adequate.
+
+**Module structure:** Clean separation of concerns. `web/` for transport, `protocol/` for parsing/validation, `api/` for business logic, `storage/` for persistence with adapter pattern, `policy/` for authorisation. 99% of modules have `@moduledoc` and `@spec` annotations.
+
+**Findings:**
+- `connection.ex` is 1,925 lines — large but cohesive (per-connection state machine). Consider extracting queue management into a submodule in future.
+- `storage/adapters/postgres/events.ex` is the heaviest module — handles normalisation, queries, and tag indexing. Well-factored internally.
+
+---
+
+### 2.2 Nostr Protocol Correctness (NIPs Compliance)
+
+**Rating: ✅ Good**
+
+**NIPs implemented (advertised via NIP-11):**
+NIP-1, NIP-9, NIP-11, NIP-13, NIP-17, NIP-40, NIP-42, NIP-44, NIP-45, NIP-50, NIP-59, NIP-62, NIP-70, NIP-77, NIP-86, NIP-98. Conditional: NIP-43, NIP-66.
+
+**NIP-01 (Core Protocol):**
+- Event structure: 7 required fields validated (`id`, `pubkey`, `created_at`, `kind`, `tags`, `content`, `sig`)
+- Event ID: SHA-256 over canonical JSON serialisation `[0, pubkey, created_at, kind, tags, content]` — verified in `validate_id_hash/1`
+- Signature: BIP-340 Schnorr via `lib_secp256k1` — `Secp256k1.schnorr_valid?(id_bin, sig_bin, pubkey_bin)`
+- Filters: `ids`, `authors`, `kinds`, `since`, `until`, `limit`, `search`, `#<letter>` tag filters — all implemented
+- Messages: EVENT, REQ, CLOSE, NOTICE, OK, EOSE, COUNT, AUTH — all implemented with correct response format
+- Subscription IDs: validated as non-empty strings, max 64 chars
+
+**Event ID verification:** Correct. Computes SHA-256 over `JSON.encode!([0, pubkey_hex, created_at, kind, tags, content])` and compares against claimed `id`. Binary decoding verified.
+
+**Signature verification:** Uses `lib_secp256k1` (wrapper around Bitcoin Core's libsecp256k1). Schnorr verification correct per BIP-340. Can be disabled via feature flag (appropriate for testing/development).
+
+**Malformed event rejection:** Comprehensive. Invalid hex, wrong byte lengths, future timestamps (>15 min), non-integer kinds, non-string content, non-array tags — all produce specific error atoms returned via OK message with `false` status.
+
+**Filter application:** Correct implementation. Each dimension (ids, authors, kinds, since, until, tags, search) is an independent predicate; all must match for a filter to match. Multiple filters are OR'd together (`matches_any?`).
+
+**Spec deviations:**
+- Tag filter values capped at 128 per filter (configurable) — stricter than spec but reasonable
+- Max 16 filters per REQ (configurable) — reasonable relay policy
+- Max 256 tags per event — reasonable relay policy
+- Search is substring/FTS, not regex — spec doesn't mandate regex
+
+---
+
+### 2.3 WebSocket Handling
+
+**Rating: ⚠️ Needs Work**
+
+**Backpressure: ✅ Excellent.**
+Three-tier strategy with configurable overflow behaviour:
+1. `:close` (default) — closes connection on queue overflow with NOTICE
+2. `:drop_newest` — silently drops incoming events
+3. `:drop_oldest` — drops oldest queued event, enqueues new
+
+Queue depth defaults: max 256, drain batch size 64. Pressure monitoring at 75% threshold emits telemetry. Batch draining prevents thundering herd.
+
+**Connection cleanup: ✅ Solid.**
+`terminate/1` removes subscriptions from global index, unsubscribes from streams, clears auth challenges, decrements connection stats. Process monitors in Index/Challenges/Sessions provide backup cleanup on unexpected exits.
+
+**Message size limits: ✅ Enforced.**
+- Frame size: 1MB default (`max_frame_bytes`), checked before JSON parsing
+- Event size: 256KB default (`max_event_bytes`), checked before DB write
+- Both configurable via environment variables
+
+**Per-connection subscription limit: ✅ Enforced.**
+Default 32 subscriptions per connection. Checked before opening new REQ. Returns CLOSED with `rate-limited:` prefix.
+
+**Ping/pong keepalives: ⚠️ Not implemented (optional).**
+No server-initiated WebSocket PING frames. RFC 6455 §5.5.2 makes PING optional ("MAY be sent"), and NIP-01 does not require keepalives. Bandit correctly responds to client-initiated PINGs per spec. However, server-side pings are a production best practice for detecting dead connections behind NAT/proxies — without them, the server relies on TCP-level detection which can take minutes.
+
+**Per-connection event ingest rate limiting: ✅ Implemented.**
+Default 120 events/second per connection with sliding window. Per-IP limiting at 1,000 events/second. Relay-wide cap at 10,000 events/second.
+
+---
+
+### 2.4 PostgreSQL / Database Layer
+
+**Rating: ✅ Good**
+
+**Schema design:**
+- Events table range-partitioned by `created_at` (monthly partitions)
+- Normalised `event_tags` table with composite FK to events
+- JSONB `tags` column for efficient serialisation (denormalised copy of normalised tags)
+- Binary storage for pubkeys/IDs/sigs with CHECK constraints on byte lengths
+
+**Indexes:** Comprehensive and appropriate:
+- `events(kind, created_at DESC)` — kind queries
+- `events(pubkey, created_at DESC)` — author queries
+- `events(created_at DESC)` — time-range queries
+- `events(id)` — direct lookup
+- `events(expires_at) WHERE expires_at IS NOT NULL` — expiration pruning
+- `event_tags(name, value, event_created_at DESC)` — tag filtering
+- GIN index on `to_tsvector('simple', content)` — full-text search
+- GIN trigram index on `content` — fuzzy search
+
+**Tag queries:** Efficient. Uses `EXISTS (SELECT 1 FROM event_tags ...)` subqueries with parameterised values. Primary tag filter applied first, remaining filters chained. No N+1 — bulk tag insert via `Repo.insert_all`.
+
+**Connection pools:** Write pool (32 default) and optional separate read pool (32 default). Queue target/interval configured at 1000ms/5000ms. Pool sizing configurable via environment variables.
+
+**Bounded results:** `max_filter_limit` defaults to 500 in config. Applied at query level via `LIMIT` clause. Post-query `Enum.take` as safety net. The `effective_filter_limit/2` function can return `nil` if both filter and opts lack a limit — but the standard API path always passes `max_filter_limit` from config.
+
+**Event expiration:** NIP-40 `expiration` tags extracted during normalisation. `ExpirationWorker` runs every 30 seconds, executing `DELETE FROM events WHERE expires_at IS NOT NULL AND expires_at <= now()`. Index on `expires_at` makes this efficient.
+
+**Partition management:** `PartitionRetentionWorker` ensures partitions exist 2 months ahead, drops oldest partitions based on configurable retention window and max DB size. Limited to 1 drop per run to avoid I/O spikes. DDL operations use `CREATE TABLE IF NOT EXISTS` and validate identifiers against `^[a-zA-Z_][a-zA-Z0-9_]*$` to prevent SQL injection.
+
+**Migrations:** Non-destructive. No `DROP COLUMN` or lock-heavy operations on existing tables. Additive indexes use `CREATE INDEX IF NOT EXISTS`. JSONB column added with data backfill in separate migration.
+
+**Findings:**
+- Multi-filter queries use union + in-memory deduplication rather than SQL `UNION`. For queries with many filters returning large result sets, this could spike memory. Acceptable for beta given the 500-result default limit.
+- Replaceable/addressable state upserts use raw SQL CTEs — parameterised, no injection risk, but harder to maintain than Ecto queries.
+
+---
+
+### 2.5 Security
+
+**Rating: ⚠️ Needs Work (one item)**
+
+**Input validation: ✅ Thorough.**
+12-step validation pipeline for events. Hex decoding with explicit byte lengths. Tag structure validation. Content type checking. All before any DB write.
+
+**Rate limiting: ✅ Well-designed.**
+Three tiers: per-connection (120/s), per-IP (1,000/s), relay-wide (10,000/s). ETS-backed with atomic counters. Configurable via environment variables. Telemetry on rate limit hits.
+
+**SQL injection: ✅ No risks found.**
+All queries use Ecto parameterisation or `fragment` with positional placeholders. Raw SQL uses `$1, $2, ...` params. Partition identifier creation validates against regex before string interpolation.
+
+**Amplification protection: ✅ Adequate.**
+- Giftwrap (kind 1059) queries blocked for unauthenticated users, restricted to recipients for authenticated users
+- `max_filter_limit: 500` bounds result sets
+- Frame/event size limits prevent memory exhaustion
+- Rate limiting prevents query flooding
+
+**NIP-42 authentication: ⚠️ Minor issue.**
+Challenge generation uses `:crypto.strong_rand_bytes(16)` — correct. Challenge stored per-connection with process monitor cleanup — correct. However, challenge comparison uses standard Erlang binary equality (`==`), not constant-time comparison. While the 16-byte random challenge makes timing attacks impractical, using `Plug.Crypto.secure_compare/2` would be best practice.
+
+**NIP-98 HTTP auth: ✅ Solid.**
+Base64-decoded event validated for kind (27235), freshness (±60s), method binding, URL binding, and signature. Replay cache prevents token reuse within TTL window.
+
+**Moderation: ✅ Complete.**
+Banned pubkeys, allowed pubkeys, banned events, blocked IPs. ETS cache with lazy loading from PostgreSQL. ACL rules with principal/capability/match structure. Protected event enforcement (NIP-70).
+
+---
+
+### 2.6 Observability & Operability
+
+**Rating: ✅ Good**
+
+**Telemetry: ✅ Excellent.**
+25+ metrics covering ingest (count, duration, outcomes), queries (count, duration, result size), fanout (duration, batch size), connections (queue depth, pressure, overflow), rate limiting (hits by scope), process health (mailbox depth), database (queue time, query time, decode time), maintenance (expiration, partition retention), and VM memory.
+
+**Prometheus endpoint: ✅ Implemented.**
+`/metrics` endpoint with access control. Histogram buckets configured for each metric type. Tagged metrics for cardinality management.
+
+**Health check: ✅ Implemented.**
+`/ready` endpoint checks Subscriptions.Index, Auth.Challenges, Negentropy.Sessions (if enabled), and all PostgreSQL repos.
+
+**Configuration: ✅ Fully externalised.**
+100+ config keys with environment variable overrides in `config/runtime.exs`. Helpers for `int_env`, `bool_env`, `csv_env`, `json_env`, `infinity_or_int_env`, `ipv4_env`. No hardcoded ports, DB URLs, or limits in application code.
+
+**Documentation: ✅ Strong.**
+99% of modules have `@moduledoc`. Public APIs have `@spec` annotations. Error types defined as type unions. Architecture docs in `docs/` covering clustering, sync, and local API.
+
+**Logging: ⚠️ Functional but basic.**
+Uses standard `Logger.error/warning` with `inspect()` formatting. No structured/JSON logging. Adequate for beta, but not ideal for log aggregation platforms.
+
+---
+
+### 2.7 Testing
+
+**Rating: ✅ Good**
+
+**Test suite:** 58 test files, ~8,000 lines of code.
+
+**Unit tests:** Protocol validation, event validator (including signature verification), filter matching, auth challenges, NIP-98 replay cache, connection policy, event policy, config, negentropy engine/message/sessions.
+
+**Integration tests:** 36 files using PostgreSQL sandbox. Event lifecycle (insert, query, update, delete), adapter contract tests, query plan regression tests, partition management, binary identifier constraints.
+
+**Protocol edge cases tested:**
+- Invalid Schnorr signatures (when verification enabled)
+- Malformed JSON → `:invalid_json`
+- Invalid event structure → specific error atoms
+- Unknown filter keys → `:invalid_filter_key`
+- Tag filter value limits (128 max)
+- NIP-43 malformed relay access events, stale join requests
+- Marmot-specific validation (encoding tags, base64 content)
+- NIP-66 discovery event validation
+
+**E2E tests:** WebSocket connection tests, TLS E2E, NAK CLI conformance, proxy IP extraction.
+
+**Load test:** `LoadSoakTest` verifying p95 fanout latency under 25ms.
+
+**Property-based tests: ⚠️ Minimal.**
+Single file (`FilterPropertyTest`) using `stream_data` for author filter membership. `stream_data` is a dependency but barely used. Significant opportunity to add property tests for event ID computation, filter boundary conditions, and tag parsing.
+
+**Missing coverage:**
+- Cluster failover / multi-node crash recovery
+- Connection pool exhaustion under load
+- WebSocket frame fragmentation
+- Concurrent subscription mutation stress
+- Byzantine negentropy scenarios
+
+---
+
+### 2.8 Dependencies
+
+**Rating: ✅ Good**
+
+| Dependency | Version | Status |
+|---|---|---|
+| `bandit` | 1.10.3 | Current, actively maintained |
+| `plug` | 1.19.1 | Current, security-patched |
+| `ecto_sql` | 3.13.5 | Current |
+| `postgrex` | 0.22.0 | Current |
+| `lib_secp256k1` | 0.7.1 | Stable, wraps Bitcoin Core's libsecp256k1 |
+| `req` | 0.5.17 | Current |
+| `telemetry_metrics_prometheus` | 1.1.0 | Current |
+| `websockex` | 0.4.x | Test-only, stable |
+| `stream_data` | 1.3.0 | Current |
+| `credo` | 1.7.x | Dev-only, current |
+
+**Cryptographic library assessment:** `lib_secp256k1` wraps libsecp256k1, the battle-tested C library from Bitcoin Core. Used only for Schnorr signature verification (BIP-340). Appropriate and trustworthy.
+
+**No outdated or unmaintained dependencies.** No known CVEs in current dependency versions.
+
+---
+
+## 3. Top 5 Issues to Fix Before Beta
+
+### 1. Add WebSocket Ping/Pong Keepalives
+**Severity:** High
+**Impact:** Long-lived subscriptions through proxies/NAT silently disconnect; server accumulates dead connections and leaked subscriptions until process monitor triggers (which requires the TCP connection to fully close).
+**Fix:** Implement periodic WebSocket PING frames (e.g., every 30s) in the connection handler. Close connections that don't respond within a timeout.
+**Files:** `lib/parrhesia/web/connection.ex`
+
+### 2. Use Constant-Time Comparison for NIP-42 Challenges
+**Severity:** Medium (low practical risk due to 16-byte random challenge, but best practice)
+**Impact:** Theoretical timing side-channel on challenge validation.
+**Fix:** Replace `challenge == stored_challenge` with `Plug.Crypto.secure_compare(challenge, stored_challenge)` in `Auth.Challenges`.
+**Files:** `lib/parrhesia/auth/challenges.ex`
+
+### 3. Expand Property-Based Testing
+**Severity:** Medium
+**Impact:** Undiscovered edge cases in event validation, filter matching, and tag parsing. `stream_data` is already a dependency but only used in one test file.
+**Fix:** Add property tests for: event ID computation with random payloads, filter boundary conditions (since/until edge cases), tag parsing with adversarial input, subscription index correctness under random insert/delete sequences.
+**Files:** `test/parrhesia/protocol/`
+
+### 4. Add Structured Logging
+**Severity:** Low-Medium
+**Impact:** Log aggregation (ELK, Datadog, Grafana Loki) requires structured output. Current plaintext logs are adequate for development but make production debugging harder at scale.
+**Fix:** Add JSON log formatter (e.g., `LoggerJSON` or custom formatter). Include connection ID, subscription ID, and event kind as structured fields.
+**Files:** `config/config.exs`, new formatter module
+
+### 5. Add Server-Initiated WebSocket PING Frames
+**Severity:** Low (not spec-required)
+**Impact:** RFC 6455 §5.5.2 makes PING optional, and NIP-01 does not require keepalives. However, without server-initiated pings, dead connections behind NAT/proxies are only detected via TCP-level timeouts (which can take minutes), during which subscriptions and state remain allocated.
+**Fix:** Consider periodic PING frames (e.g., every 30s) in the connection handler to proactively detect dead connections.
+**Files:** `lib/parrhesia/web/connection.ex`
+
+---
+
+## 4. Nice-to-Haves for Beta
+
+1. **Extract queue management** from `connection.ex` (1,925 lines) into a dedicated `Parrhesia.Web.OutboundQueue` module for maintainability.
+
+2. **Add request correlation IDs** to WebSocket connections for log tracing across the event lifecycle (ingest → validation → storage → fanout).
+
+3. **SQL UNION for multi-filter queries** instead of in-memory deduplication. Would reduce memory spikes for queries with many filters, though the 500-result limit mitigates this.
+
+4. **Slow query telemetry** — flag database queries exceeding a configurable threshold (e.g., 100ms) via dedicated telemetry event.
+
+5. **Connection idle timeout** — close WebSocket connections with no activity for a configurable period (e.g., 30 minutes), independent of ping/pong.
+
+6. **Per-pubkey rate limiting** — current rate limiting is per-connection and per-IP. A determined attacker could use multiple IPs. Per-pubkey limiting would add a layer but requires NIP-42 auth.
+
+7. **OpenTelemetry integration** — for distributed tracing across multi-relay deployments.
+
+8. **Negentropy session memory bounds** — while capped at 10K sessions, large filter sets within sessions could still consume significant memory. Consider per-session size limits.
+
+---
+
+## 5. Promotion Recommendation
+
+### ⚠️ Promote with Conditions
+
+**The codebase meets beta promotion criteria with one condition:**
+
+**Condition: Use constant-time comparison for NIP-42 challenge validation.**
+While the practical risk is low (16-byte random challenge), this is a security best practice that takes one line to fix (`Plug.Crypto.secure_compare/2`).
+
+**Criteria assessment:**
+
+| Criterion | Status |
+|---|---|
+| NIP-01 correctly and fully implemented | ✅ Yes |
+| Event signature and ID verification cryptographically correct | ✅ Yes (lib_secp256k1 / BIP-340) |
+| No ❌ blocking in Security | ✅ No blockers (one minor item) |
+| No ❌ blocking in WebSocket handling | ✅ No blockers (ping/pong optional per RFC 6455) |
+| System does not crash or leak resources under normal load | ✅ Process monitors, queue management, rate limiting |
+| Working test suite covering critical protocol path | ✅ 58 files, 8K LOC, protocol edge cases covered |
+| Basic operability (config externalised, logs meaningful) | ✅ 100+ config keys, telemetry, Prometheus, health check |
+
+**What's strong:**
+- OTP design is production-grade
+- Protocol implementation is comprehensive and spec-compliant
+- Security posture is solid (rate limiting, input validation, ACLs, moderation)
+- Telemetry coverage is excellent
+- Test suite covers critical paths
+- Dependencies are current and trustworthy
+
+**What needs attention post-beta:**
+- Expand property-based testing
+- Add structured logging for production observability
+- Consider per-pubkey rate limiting
+- Connection handler module size (1,925 lines)

flake.nix

@@ -18,12 +18,18 @@
     packages = forAllSystems (
       system: let
         pkgs = import nixpkgs {inherit system;};
+        pkgsLinux = import nixpkgs {system = "x86_64-linux";};
         lib = pkgs.lib;
         parrhesia = pkgs.callPackage ./default.nix {};
+        nostrBench = pkgs.callPackage ./nix/nostr-bench.nix {};
       in
         {
           default = parrhesia;
-          inherit parrhesia;
+          inherit parrhesia nostrBench;
+
+          # Uses x86_64-linux pkgs so it can cross-build via a remote
+          # builder even when the host is aarch64-darwin.
+          nostrBenchStaticX86_64Musl = pkgsLinux.callPackage ./nix/nostr-bench.nix {staticX86_64Musl = true;};
         }
         // lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux {
           dockerImage = pkgs.dockerTools.buildLayeredImage {

justfile

@@ -0,0 +1,79 @@
+set shell := ["bash", "-euo", "pipefail", "-c"]
+
+repo_root := justfile_directory()
+
+# Show curated command help (same as `just help`).
+default:
+  @just help
+
+# Show top-level or topic-specific help.
+help topic="":
+  @cd "{{repo_root}}" && ./scripts/just_help.sh "{{topic}}"
+
+# Raw e2e harness commands.
+e2e subcommand *args:
+  #!/usr/bin/env bash
+  set -euo pipefail
+  cd "{{repo_root}}"
+  subcommand="{{subcommand}}"
+
+  if [[ -z "$subcommand" || "$subcommand" == "help" ]]; then
+    just help e2e
+  elif [[ "$subcommand" == "nak" ]]; then
+    ./scripts/run_nak_e2e.sh {{args}}
+  elif [[ "$subcommand" == "marmot" ]]; then
+    ./scripts/run_marmot_e2e.sh {{args}}
+  elif [[ "$subcommand" == "node-sync" ]]; then
+    ./scripts/run_node_sync_e2e.sh {{args}}
+  elif [[ "$subcommand" == "node-sync-docker" ]]; then
+    ./scripts/run_node_sync_docker_e2e.sh {{args}}
+  elif [[ "$subcommand" == "suite" ]]; then
+    if [[ -z "{{args}}" ]]; then
+      echo "usage: just e2e suite <suite-name> <command> [args...]" >&2
+      exit 1
+    fi
+    ./scripts/run_e2e_suite.sh {{args}}
+  else
+    echo "Unknown e2e subcommand: $subcommand" >&2
+    just help e2e
+    exit 1
+  fi
+
+# Benchmark flows (local/cloud/history + direct relay targets).
+bench subcommand *args:
+  #!/usr/bin/env bash
+  set -euo pipefail
+  cd "{{repo_root}}"
+  subcommand="{{subcommand}}"
+
+  if [[ -z "$subcommand" || "$subcommand" == "help" ]]; then
+    just help bench
+  elif [[ "$subcommand" == "compare" ]]; then
+    ./scripts/run_bench_compare.sh {{args}}
+  elif [[ "$subcommand" == "collect" ]]; then
+    ./scripts/run_bench_collect.sh {{args}}
+  elif [[ "$subcommand" == "update" ]]; then
+    ./scripts/run_bench_update.sh {{args}}
+  elif [[ "$subcommand" == "list" ]]; then
+    ./scripts/run_bench_update.sh --list {{args}}
+  elif [[ "$subcommand" == "at" ]]; then
+    if [[ -z "{{args}}" ]]; then
+      echo "usage: just bench at <git-ref>" >&2
+      exit 1
+    fi
+    ./scripts/run_bench_at_ref.sh {{args}}
+  elif [[ "$subcommand" == "cloud" ]]; then
+    ./scripts/run_bench_cloud.sh {{args}}
+  elif [[ "$subcommand" == "cloud-quick" ]]; then
+    ./scripts/run_bench_cloud.sh --quick {{args}}
+  elif [[ "$subcommand" == "relay" ]]; then
+    ./scripts/run_nostr_bench.sh {{args}}
+  elif [[ "$subcommand" == "relay-strfry" ]]; then
+    ./scripts/run_nostr_bench_strfry.sh {{args}}
+  elif [[ "$subcommand" == "relay-nostr-rs" ]]; then
+    ./scripts/run_nostr_bench_nostr_rs_relay.sh {{args}}
+  else
+    echo "Unknown bench subcommand: $subcommand" >&2
+    just help bench
+    exit 1
+  fi

lib/parrhesia.ex

@@ -3,6 +3,7 @@ defmodule Parrhesia do
   Parrhesia is a Nostr relay runtime that can run standalone or as an embedded OTP service.
 
   For embedded use, the main developer-facing surface is `Parrhesia.API.*`.
+  For host-managed HTTP/WebSocket ingress mounting, use `Parrhesia.Plug`.
   Start with:
 
   - `Parrhesia.API.Events`
@@ -11,6 +12,7 @@ defmodule Parrhesia do
   - `Parrhesia.API.Identity`
   - `Parrhesia.API.ACL`
   - `Parrhesia.API.Sync`
+  - `Parrhesia.Plug`
 
   The host application is responsible for:
 

lib/parrhesia/auth/challenges.ex

@@ -67,7 +67,16 @@ defmodule Parrhesia.Auth.Challenges do
   end
 
   def handle_call({:valid?, owner_pid, challenge}, _from, state) do
-    {:reply, Map.get(state.entries, owner_pid) == challenge, state}
+    valid? =
+      case Map.get(state.entries, owner_pid) do
+        stored_challenge when is_binary(stored_challenge) ->
+          Plug.Crypto.secure_compare(stored_challenge, challenge)
+
+        _other ->
+          false
+      end
+
+    {:reply, valid?, state}
   end
 
   def handle_call({:clear, owner_pid}, _from, state) do

lib/parrhesia/plug.ex

@@ -0,0 +1,113 @@
+defmodule Parrhesia.Plug do
+  @moduledoc """
+  Official Plug interface for mounting Parrhesia HTTP/WebSocket ingress in a host app.
+
+  This plug serves the same route surface as the built-in listener endpoint:
+
+  - `GET /health`
+  - `GET /ready`
+  - `GET /relay` (NIP-11 + websocket transport)
+  - `POST /management`
+  - `GET /metrics`
+
+  ## Options
+
+    * `:listener` - listener configuration used to authorize and serve requests.
+      Supported values:
+      * an atom listener id from `config :parrhesia, :listeners` (for example `:public`)
+      * a listener config map/keyword list (same schema as `:listeners` entries)
+
+  When a host app owns the HTTPS edge, a common pattern is:
+
+      config :parrhesia, :listeners, %{}
+
+  and mount `Parrhesia.Plug` with an explicit `:listener` map.
+  """
+
+  @behaviour Plug
+
+  alias Parrhesia.Web.Listener
+  alias Parrhesia.Web.Router
+
+  @type listener_option :: atom() | map() | keyword()
+  @type option :: {:listener, listener_option()}
+
+  @spec init([option()]) :: keyword()
+  @impl Plug
+  def init(opts) do
+    opts = Keyword.validate!(opts, listener: :public)
+    listener = opts |> Keyword.fetch!(:listener) |> resolve_listener!()
+    [listener: listener]
+  end
+
+  @spec call(Plug.Conn.t(), keyword()) :: Plug.Conn.t()
+  @impl Plug
+  def call(conn, opts) do
+    conn
+    |> Listener.put_conn(opts)
+    |> Router.call([])
+  end
+
+  defp resolve_listener!(listener_id) when is_atom(listener_id) do
+    listeners = Application.get_env(:parrhesia, :listeners, %{})
+
+    case lookup_listener_by_id(listeners, listener_id) do
+      nil ->
+        raise ArgumentError,
+              "listener #{inspect(listener_id)} not found in config :parrhesia, :listeners; " <>
+                "configure it there or pass :listener as a map"
+
+      listener ->
+        Listener.from_opts(listener: listener)
+    end
+  end
+
+  defp resolve_listener!(listener) when is_map(listener) do
+    Listener.from_opts(listener: listener)
+  end
+
+  defp resolve_listener!(listener) when is_list(listener) do
+    if Keyword.keyword?(listener) do
+      Listener.from_opts(listener: Map.new(listener))
+    else
+      raise ArgumentError,
+            ":listener keyword list must be a valid keyword configuration"
+    end
+  end
+
+  defp resolve_listener!(other) do
+    raise ArgumentError,
+          ":listener must be an atom id, map, or keyword list, got: #{inspect(other)}"
+  end
+
+  defp lookup_listener_by_id(listeners, listener_id) when is_map(listeners) do
+    case Map.fetch(listeners, listener_id) do
+      {:ok, listener} when is_map(listener) ->
+        Map.put_new(listener, :id, listener_id)
+
+      {:ok, listener} when is_list(listener) ->
+        listener |> Map.new() |> Map.put_new(:id, listener_id)
+
+      _other ->
+        nil
+    end
+  end
+
+  defp lookup_listener_by_id(listeners, listener_id) when is_list(listeners) do
+    case Enum.find(listeners, fn
+           {id, _listener} -> id == listener_id
+           _other -> false
+         end) do
+      {^listener_id, listener} when is_map(listener) ->
+        Map.put_new(listener, :id, listener_id)
+
+      {^listener_id, listener} when is_list(listener) ->
+        listener |> Map.new() |> Map.put_new(:id, listener_id)
+
+      _other ->
+        nil
+    end
+  end
+
+  defp lookup_listener_by_id(_listeners, _listener_id), do: nil
+end

lib/parrhesia/storage/adapters/memory/store.ex

@@ -22,7 +22,17 @@ defmodule Parrhesia.Storage.Adapters.Memory.Store do
     audit_logs: []
   }
 
-  def ensure_started, do: start_store()
+  def start_link(opts \\ []) do
+    name = Keyword.get(opts, :name, @name)
+    Agent.start_link(&init_state/0, name: name)
+  end
+
+  def ensure_started do
+    case Process.whereis(@name) do
+      pid when is_pid(pid) -> :ok
+      nil -> start_store()
+    end
+  end
 
   def put_event(event_id, event) when is_binary(event_id) and is_map(event) do
     :ok = ensure_started()
@@ -159,28 +169,72 @@ defmodule Parrhesia.Storage.Adapters.Memory.Store do
   defp normalize_reduce_result(next_acc), do: next_acc
 
   def get(fun) do
-    :ok = ensure_started()
-    Agent.get(@name, fun)
+    with_store(fn pid -> Agent.get(pid, fun) end)
   end
 
   def update(fun) do
-    :ok = ensure_started()
-    Agent.update(@name, fun)
+    with_store(fn pid -> Agent.update(pid, fun) end)
   end
 
   def get_and_update(fun) do
-    :ok = ensure_started()
-    Agent.get_and_update(@name, fun)
+    with_store(fn pid -> Agent.get_and_update(pid, fun) end)
   end
 
   defp start_store do
-    case Agent.start_link(&init_state/0, name: @name) do
-      {:ok, _pid} -> :ok
-      {:error, {:already_started, _pid}} -> :ok
-      {:error, reason} -> {:error, reason}
+    case start_link() do
+      {:ok, _pid} ->
+        :ok
+
+      {:error, {:already_started, pid}} ->
+        if Process.alive?(pid) do
+          :ok
+        else
+          wait_for_store_exit(pid)
+        end
+
+      {:error, reason} ->
+        {:error, reason}
     end
   end
 
+  defp with_store(fun, attempts \\ 2)
+
+  defp with_store(fun, attempts) when attempts > 0 do
+    :ok = ensure_started()
+
+    case Process.whereis(@name) do
+      pid when is_pid(pid) ->
+        try do
+          fun.(pid)
+        catch
+          :exit, reason ->
+            if noproc_exit?(reason) and attempts > 1 do
+              with_store(fun, attempts - 1)
+            else
+              exit(reason)
+            end
+        end
+
+      nil ->
+        with_store(fun, attempts - 1)
+    end
+  end
+
+  defp with_store(_fun, 0), do: exit(:noproc)
+
+  defp wait_for_store_exit(pid) do
+    ref = Process.monitor(pid)
+
+    receive do
+      {:DOWN, ^ref, :process, ^pid, _reason} -> start_store()
+    after
+      100 -> start_store()
+    end
+  end
+
+  defp noproc_exit?({:noproc, _details}), do: true
+  defp noproc_exit?(_reason), do: false
+
   defp init_state do
     ensure_tables_started()
 

lib/parrhesia/storage/supervisor.ex

@@ -13,11 +13,19 @@ defmodule Parrhesia.Storage.Supervisor do
 
   @impl true
   def init(_init_arg) do
-    children = moderation_cache_children() ++ PostgresRepos.started_repos()
+    children =
+      memory_store_children() ++ moderation_cache_children() ++ PostgresRepos.started_repos()
 
     Supervisor.init(children, strategy: :one_for_one)
   end
 
+  defp memory_store_children do
+    case Application.get_env(:parrhesia, :storage, [])[:backend] do
+      :memory -> [Parrhesia.Storage.Adapters.Memory.Store]
+      _other -> []
+    end
+  end
+
   defp moderation_cache_children do
     if PostgresRepos.postgres_enabled?() and
          Application.get_env(:parrhesia, :moderation_cache_enabled, true) do

lib/parrhesia/test_support/tls_certs.ex

@@ -137,9 +137,16 @@ defmodule Parrhesia.TestSupport.TLSCerts do
   end
 
   defp openssl!(args) do
-    case System.cmd("/usr/bin/openssl", args, stderr_to_stdout: true) do
+    case System.cmd(openssl_executable!(), args, stderr_to_stdout: true) do
       {output, 0} -> output
       {output, status} -> raise "openssl failed with status #{status}: #{output}"
     end
   end
+
+  defp openssl_executable! do
+    case System.find_executable("openssl") do
+      nil -> raise "openssl executable not found in PATH"
+      path -> path
+    end
+  end
 end

lib/parrhesia/web/connection.ex

@@ -30,7 +30,11 @@ defmodule Parrhesia.Web.Connection do
   @default_event_ingest_rate_limit 120
   @default_event_ingest_window_seconds 1
   @default_auth_max_age_seconds 600
+  @default_websocket_ping_interval_seconds 30
+  @default_websocket_pong_timeout_seconds 10
   @drain_outbound_queue :drain_outbound_queue
+  @websocket_keepalive_ping :websocket_keepalive_ping
+  @websocket_keepalive_timeout :websocket_keepalive_timeout
   @outbound_queue_pressure_threshold 0.75
 
   @marmot_kinds MapSet.new([
@@ -72,6 +76,10 @@ defmodule Parrhesia.Web.Connection do
             event_ingest_window_started_at_ms: 0,
             event_ingest_count: 0,
             auth_max_age_seconds: @default_auth_max_age_seconds,
+            websocket_ping_interval_seconds: @default_websocket_ping_interval_seconds,
+            websocket_pong_timeout_seconds: @default_websocket_pong_timeout_seconds,
+            websocket_keepalive_timeout_timer_ref: nil,
+            websocket_awaiting_pong_payload: nil,
             track_population?: true
 
   @type overflow_strategy :: :close | :drop_oldest | :drop_newest
@@ -109,6 +117,10 @@ defmodule Parrhesia.Web.Connection do
           event_ingest_window_started_at_ms: integer(),
           event_ingest_count: non_neg_integer(),
           auth_max_age_seconds: pos_integer(),
+          websocket_ping_interval_seconds: non_neg_integer(),
+          websocket_pong_timeout_seconds: pos_integer(),
+          websocket_keepalive_timeout_timer_ref: reference() | nil,
+          websocket_awaiting_pong_payload: binary() | nil,
           track_population?: boolean()
         }
 
@@ -117,29 +129,33 @@ defmodule Parrhesia.Web.Connection do
     maybe_configure_exit_trapping(opts)
     auth_challenges = auth_challenges(opts)
 
-    state = %__MODULE__{
-      listener: Listener.from_opts(opts),
-      transport_identity: transport_identity(opts),
-      max_subscriptions_per_connection: max_subscriptions_per_connection(opts),
-      subscription_index: subscription_index(opts),
-      auth_challenges: auth_challenges,
-      auth_challenge: maybe_issue_auth_challenge(auth_challenges),
-      relay_url: relay_url(opts),
-      remote_ip: remote_ip(opts),
-      negentropy_sessions: negentropy_sessions(opts),
-      max_outbound_queue: max_outbound_queue(opts),
-      outbound_overflow_strategy: outbound_overflow_strategy(opts),
-      outbound_drain_batch_size: outbound_drain_batch_size(opts),
-      max_frame_bytes: max_frame_bytes(opts),
-      max_event_bytes: max_event_bytes(opts),
-      event_ingest_limiter: event_ingest_limiter(opts),
-      remote_ip_event_ingest_limiter: remote_ip_event_ingest_limiter(opts),
-      max_event_ingest_per_window: max_event_ingest_per_window(opts),
-      event_ingest_window_seconds: event_ingest_window_seconds(opts),
-      event_ingest_window_started_at_ms: System.monotonic_time(:millisecond),
-      auth_max_age_seconds: auth_max_age_seconds(opts),
-      track_population?: track_population?(opts)
-    }
+    state =
+      %__MODULE__{
+        listener: Listener.from_opts(opts),
+        transport_identity: transport_identity(opts),
+        max_subscriptions_per_connection: max_subscriptions_per_connection(opts),
+        subscription_index: subscription_index(opts),
+        auth_challenges: auth_challenges,
+        auth_challenge: maybe_issue_auth_challenge(auth_challenges),
+        relay_url: relay_url(opts),
+        remote_ip: remote_ip(opts),
+        negentropy_sessions: negentropy_sessions(opts),
+        max_outbound_queue: max_outbound_queue(opts),
+        outbound_overflow_strategy: outbound_overflow_strategy(opts),
+        outbound_drain_batch_size: outbound_drain_batch_size(opts),
+        max_frame_bytes: max_frame_bytes(opts),
+        max_event_bytes: max_event_bytes(opts),
+        event_ingest_limiter: event_ingest_limiter(opts),
+        remote_ip_event_ingest_limiter: remote_ip_event_ingest_limiter(opts),
+        max_event_ingest_per_window: max_event_ingest_per_window(opts),
+        event_ingest_window_seconds: event_ingest_window_seconds(opts),
+        event_ingest_window_started_at_ms: System.monotonic_time(:millisecond),
+        auth_max_age_seconds: auth_max_age_seconds(opts),
+        websocket_ping_interval_seconds: websocket_ping_interval_seconds(opts),
+        websocket_pong_timeout_seconds: websocket_pong_timeout_seconds(opts),
+        track_population?: track_population?(opts)
+      }
+      |> maybe_schedule_next_websocket_ping()
 
     :ok = maybe_track_connection_open(state)
     Telemetry.emit_process_mailbox_depth(:connection)
@@ -180,6 +196,17 @@ defmodule Parrhesia.Web.Connection do
     |> emit_connection_mailbox_depth()
   end
 
+  @impl true
+  def handle_control({payload, [opcode: :pong]}, %__MODULE__{} = state) when is_binary(payload) do
+    {:ok, maybe_acknowledge_websocket_pong(state, payload)}
+    |> emit_connection_mailbox_depth()
+  end
+
+  def handle_control({_payload, [opcode: :ping]}, %__MODULE__{} = state) do
+    {:ok, state}
+    |> emit_connection_mailbox_depth()
+  end
+
   defp handle_decoded_message({:event, event}, state), do: handle_event_ingest(state, event)
 
   defp handle_decoded_message({:req, subscription_id, filters}, state),
@@ -291,6 +318,24 @@ defmodule Parrhesia.Web.Connection do
     end
   end
 
+  def handle_info(@websocket_keepalive_ping, %__MODULE__{} = state) do
+    state
+    |> maybe_schedule_next_websocket_ping()
+    |> maybe_send_websocket_keepalive_ping()
+    |> emit_connection_mailbox_depth()
+  end
+
+  def handle_info({@websocket_keepalive_timeout, payload}, %__MODULE__{} = state)
+      when is_binary(payload) do
+    if websocket_keepalive_timeout_payload?(state, payload) do
+      {:stop, :normal, {1001, "keepalive timeout"}, state}
+      |> emit_connection_mailbox_depth()
+    else
+      {:ok, state}
+      |> emit_connection_mailbox_depth()
+    end
+  end
+
   def handle_info({:EXIT, _from, :shutdown}, %__MODULE__{} = state) do
     close_with_drained_outbound_frames(state)
     |> emit_connection_mailbox_depth()
@@ -313,6 +358,7 @@ defmodule Parrhesia.Web.Connection do
     :ok = maybe_unsubscribe_all_stream_subscriptions(state)
     :ok = maybe_remove_index_owner(state)
     :ok = maybe_clear_auth_challenge(state)
+    :ok = cancel_websocket_keepalive_timers(state)
     :ok
   end
 
@@ -1229,6 +1275,93 @@ defmodule Parrhesia.Web.Connection do
     %__MODULE__{state | drain_scheduled?: true}
   end
 
+  defp maybe_schedule_next_websocket_ping(
+         %__MODULE__{websocket_ping_interval_seconds: interval_seconds} = state
+       )
+       when interval_seconds <= 0,
+       do: state
+
+  defp maybe_schedule_next_websocket_ping(
+         %__MODULE__{websocket_ping_interval_seconds: interval_seconds} = state
+       ) do
+    _timer_ref = Process.send_after(self(), @websocket_keepalive_ping, interval_seconds * 1_000)
+    state
+  end
+
+  defp maybe_send_websocket_keepalive_ping(
+         %__MODULE__{websocket_ping_interval_seconds: interval_seconds} = state
+       )
+       when interval_seconds <= 0,
+       do: {:ok, state}
+
+  defp maybe_send_websocket_keepalive_ping(
+         %__MODULE__{websocket_awaiting_pong_payload: awaiting_payload} = state
+       )
+       when is_binary(awaiting_payload),
+       do: {:ok, state}
+
+  defp maybe_send_websocket_keepalive_ping(%__MODULE__{} = state) do
+    payload = Base.encode16(:crypto.strong_rand_bytes(8), case: :lower)
+
+    timeout_timer_ref =
+      Process.send_after(
+        self(),
+        {@websocket_keepalive_timeout, payload},
+        state.websocket_pong_timeout_seconds * 1_000
+      )
+
+    next_state =
+      %__MODULE__{
+        state
+        | websocket_keepalive_timeout_timer_ref: timeout_timer_ref,
+          websocket_awaiting_pong_payload: payload
+      }
+
+    {:push, {:ping, payload}, next_state}
+  end
+
+  defp websocket_keepalive_timeout_payload?(
+         %__MODULE__{websocket_awaiting_pong_payload: awaiting_payload},
+         payload
+       )
+       when is_binary(awaiting_payload) and is_binary(payload) do
+    Plug.Crypto.secure_compare(awaiting_payload, payload)
+  end
+
+  defp websocket_keepalive_timeout_payload?(_state, _payload), do: false
+
+  defp maybe_acknowledge_websocket_pong(
+         %__MODULE__{websocket_awaiting_pong_payload: awaiting_payload} = state,
+         payload
+       )
+       when is_binary(awaiting_payload) and is_binary(payload) do
+    if Plug.Crypto.secure_compare(awaiting_payload, payload) do
+      :ok = cancel_timer(state.websocket_keepalive_timeout_timer_ref)
+
+      %__MODULE__{
+        state
+        | websocket_keepalive_timeout_timer_ref: nil,
+          websocket_awaiting_pong_payload: nil
+      }
+    else
+      state
+    end
+  end
+
+  defp maybe_acknowledge_websocket_pong(%__MODULE__{} = state, _payload), do: state
+
+  defp cancel_websocket_keepalive_timers(%__MODULE__{} = state) do
+    :ok = cancel_timer(state.websocket_keepalive_timeout_timer_ref)
+    :ok
+  end
+
+  defp cancel_timer(timer_ref) when is_reference(timer_ref) do
+    _ = Process.cancel_timer(timer_ref, async: true, info: false)
+    :ok
+  end
+
+  defp cancel_timer(_timer_ref), do: :ok
+
   defp emit_outbound_queue_depth(state, metadata \\ %{}) do
     depth = state.outbound_queue_size
 
@@ -1769,6 +1902,64 @@ defmodule Parrhesia.Web.Connection do
     |> Keyword.get(:auth_max_age_seconds, @default_auth_max_age_seconds)
   end
 
+  defp websocket_ping_interval_seconds(opts) when is_list(opts) do
+    opts
+    |> Keyword.get(:websocket_ping_interval_seconds)
+    |> normalize_websocket_ping_interval_seconds()
+  end
+
+  defp websocket_ping_interval_seconds(opts) when is_map(opts) do
+    opts
+    |> Map.get(:websocket_ping_interval_seconds)
+    |> normalize_websocket_ping_interval_seconds()
+  end
+
+  defp websocket_ping_interval_seconds(_opts), do: configured_websocket_ping_interval_seconds()
+
+  defp normalize_websocket_ping_interval_seconds(value)
+       when is_integer(value) and value >= 0,
+       do: value
+
+  defp normalize_websocket_ping_interval_seconds(_value),
+    do: configured_websocket_ping_interval_seconds()
+
+  defp configured_websocket_ping_interval_seconds do
+    case Application.get_env(:parrhesia, :limits, [])
+         |> Keyword.get(:websocket_ping_interval_seconds) do
+      value when is_integer(value) and value >= 0 -> value
+      _other -> @default_websocket_ping_interval_seconds
+    end
+  end
+
+  defp websocket_pong_timeout_seconds(opts) when is_list(opts) do
+    opts
+    |> Keyword.get(:websocket_pong_timeout_seconds)
+    |> normalize_websocket_pong_timeout_seconds()
+  end
+
+  defp websocket_pong_timeout_seconds(opts) when is_map(opts) do
+    opts
+    |> Map.get(:websocket_pong_timeout_seconds)
+    |> normalize_websocket_pong_timeout_seconds()
+  end
+
+  defp websocket_pong_timeout_seconds(_opts), do: configured_websocket_pong_timeout_seconds()
+
+  defp normalize_websocket_pong_timeout_seconds(value)
+       when is_integer(value) and value > 0,
+       do: value
+
+  defp normalize_websocket_pong_timeout_seconds(_value),
+    do: configured_websocket_pong_timeout_seconds()
+
+  defp configured_websocket_pong_timeout_seconds do
+    case Application.get_env(:parrhesia, :limits, [])
+         |> Keyword.get(:websocket_pong_timeout_seconds) do
+      value when is_integer(value) and value > 0 -> value
+      _other -> @default_websocket_pong_timeout_seconds
+    end
+  end
+
   defp track_population?(opts) when is_list(opts), do: Keyword.get(opts, :track_population?, true)
   defp track_population?(opts) when is_map(opts), do: Map.get(opts, :track_population?, true)
   defp track_population?(_opts), do: true

lib/parrhesia/web/listener.ex

@@ -177,7 +177,7 @@ defmodule Parrhesia.Web.Listener do
       ip: listener.bind.ip,
       port: listener.bind.port,
       scheme: scheme,
-      plug: {Parrhesia.Web.ListenerPlug, listener: listener}
+      plug: {Parrhesia.Plug, listener: listener}
     ] ++
       TLS.bandit_options(listener.transport.tls) ++
       [thousand_island_options: thousand_island_options] ++

lib/parrhesia/web/listener_plug.ex

@@ -1,14 +0,0 @@
-defmodule Parrhesia.Web.ListenerPlug do
-  @moduledoc false
-
-  alias Parrhesia.Web.Listener
-  alias Parrhesia.Web.Router
-
-  def init(opts), do: opts
-
-  def call(conn, opts) do
-    conn
-    |> Listener.put_conn(opts)
-    |> Router.call([])
-  end
-end

mix.exs

@@ -4,7 +4,7 @@ defmodule Parrhesia.MixProject do
   def project do
     [
       app: :parrhesia,
-      version: "0.6.0",
+      version: "0.7.0",
       elixir: "~> 1.18",
       elixirc_paths: elixirc_paths(Mix.env()),
       start_permanent: Mix.env() == :prod,
@@ -26,7 +26,11 @@ defmodule Parrhesia.MixProject do
   defp elixirc_paths(_env), do: ["lib"]
 
   def cli do
-    [preferred_envs: [precommit: :test, bench: :test, "bench.update": :test]]
+    [
+      preferred_envs: [
+        precommit: :test
+      ]
+    ]
   end
 
   # Run "mix help deps" to learn about dependencies.
@@ -66,13 +70,6 @@ defmodule Parrhesia.MixProject do
       "ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
       "ecto.reset": ["ecto.drop", "ecto.setup"],
       test: ["ecto.create --quiet", "ecto.migrate --quiet", "test"],
-      "test.nak_e2e": ["cmd ./scripts/run_nak_e2e.sh"],
-      "test.marmot_e2e": ["cmd ./scripts/run_marmot_e2e.sh"],
-      "test.node_sync_e2e": ["cmd ./scripts/run_node_sync_e2e.sh"],
-      "test.node_sync_docker_e2e": ["cmd ./scripts/run_node_sync_docker_e2e.sh"],
-      bench: ["cmd ./scripts/run_bench_compare.sh"],
-      "bench.update": ["cmd ./scripts/run_bench_update.sh"],
-      # cov: ["cmd mix coveralls.lcov"],
       lint: ["format --check-formatted", "credo"],
       precommit: [
         "format",
@@ -80,8 +77,7 @@ defmodule Parrhesia.MixProject do
         "credo --strict --all",
         "deps.unlock --unused",
         "test",
-        # "test.nak_e2e",
-        "test.marmot_e2e"
+        "cmd just e2e marmot"
       ]
     ]
   end
@@ -113,6 +109,7 @@ defmodule Parrhesia.MixProject do
         ],
         Runtime: [
           Parrhesia,
+          Parrhesia.Plug,
           Parrhesia.Release,
           Parrhesia.Runtime
         ]

mix.lock

@@ -17,7 +17,7 @@
   "finch": {:hex, :finch, "0.21.0", "b1c3b2d48af02d0c66d2a9ebfb5622be5c5ecd62937cf79a88a7f98d48a8290c", [:mix], [{:mime, "~> 1.0 or ~> 2.0", [hex: :mime, repo: "hexpm", optional: false]}, {:mint, "~> 1.6.2 or ~> 1.7", [hex: :mint, repo: "hexpm", optional: false]}, {:nimble_options, "~> 0.4 or ~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:nimble_pool, "~> 1.1", [hex: :nimble_pool, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "87dc6e169794cb2570f75841a19da99cfde834249568f2a5b121b809588a4377"},
   "glob_ex": {:hex, :glob_ex, "0.1.11", "cb50d3f1ef53f6ca04d6252c7fde09fd7a1cf63387714fe96f340a1349e62c93", [:mix], [], "hexpm", "342729363056e3145e61766b416769984c329e4378f1d558b63e341020525de4"},
   "hpax": {:hex, :hpax, "1.0.3", "ed67ef51ad4df91e75cc6a1494f851850c0bd98ebc0be6e81b026e765ee535aa", [:mix], [], "hexpm", "8eab6e1cfa8d5918c2ce4ba43588e894af35dbd8e91e6e55c817bca5847df34a"},
-  "igniter": {:hex, :igniter, "0.7.4", "b5f9dd512eb1e672f1c141b523142b5b4602fcca231df5b4e362999df4b88e14", [:mix], [{:glob_ex, "~> 0.1.7", [hex: :glob_ex, repo: "hexpm", optional: false]}, {:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: false]}, {:owl, "~> 0.11", [hex: :owl, repo: "hexpm", optional: false]}, {:phx_new, "~> 1.7", [hex: :phx_new, repo: "hexpm", optional: true]}, {:req, "~> 0.5", [hex: :req, repo: "hexpm", optional: false]}, {:rewrite, ">= 1.1.1 and < 2.0.0-0", [hex: :rewrite, repo: "hexpm", optional: false]}, {:sourceror, "~> 1.4", [hex: :sourceror, repo: "hexpm", optional: false]}, {:spitfire, ">= 0.1.3 and < 1.0.0-0", [hex: :spitfire, repo: "hexpm", optional: false]}], "hexpm", "971b240ee916a06b1af56381a262d9eeaff9610eddc299d61a213cd7a9d79efd"},
+  "igniter": {:hex, :igniter, "0.7.6", "687d622c735e020f13cf480c83d0fce1cc899f4fbed547f5254b960ea82d3525", [:mix], [{:glob_ex, "~> 0.1.7", [hex: :glob_ex, repo: "hexpm", optional: false]}, {:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: false]}, {:owl, "~> 0.11", [hex: :owl, repo: "hexpm", optional: false]}, {:phx_new, "~> 1.7", [hex: :phx_new, repo: "hexpm", optional: true]}, {:req, "~> 0.5", [hex: :req, repo: "hexpm", optional: false]}, {:rewrite, ">= 1.1.1 and < 2.0.0-0", [hex: :rewrite, repo: "hexpm", optional: false]}, {:sourceror, "~> 1.4", [hex: :sourceror, repo: "hexpm", optional: false]}, {:spitfire, ">= 0.1.3 and < 1.0.0-0", [hex: :spitfire, repo: "hexpm", optional: false]}], "hexpm", "424f41a41273fce0f7424008405ee073b5bd06359ca9396e841f83a669c01619"},
   "jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"},
   "lib_secp256k1": {:hex, :lib_secp256k1, "0.7.1", "53cad778b8da3a29e453a7a477517d99fb5f13f615c8050eb2db8fd1dce7a1db", [:make, :mix], [{:elixir_make, "~> 0.9", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "78bdd3661a17448aff5aeec5ca74c8ddbc09b01f0ecfa3ba1aba3e8ae47ab2b3"},
   "makeup": {:hex, :makeup, "1.2.1", "e90ac1c65589ef354378def3ba19d401e739ee7ee06fb47f94c687016e3713d1", [:mix], [{:nimble_parsec, "~> 1.4", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "d36484867b0bae0fea568d10131197a4c2e47056a6fbe84922bf6ba71c8d17ce"},

nix/nostr-bench.nix

@@ -2,27 +2,55 @@
   lib,
   fetchFromGitHub,
   rustPlatform,
-}:
-rustPlatform.buildRustPackage rec {
-  pname = "nostr-bench";
-  version = "0.4.0";
+  pkgsCross,
+  runCommand,
+  staticX86_64Musl ? false,
+}: let
+  selectedRustPlatform =
+    if staticX86_64Musl
+    then pkgsCross.musl64.rustPlatform
+    else rustPlatform;
 
-  src = fetchFromGitHub {
+  srcBase = fetchFromGitHub {
     owner = "rnostr";
-    repo = pname;
+    repo = "nostr-bench";
     rev = "d3ab701512b7c871707b209ef3f934936e407963";
     hash = "sha256-F2qg1veO1iNlVUKf1b/MV+vexiy4Tt+w2aikDDbp7tU=";
   };
 
-  cargoHash = "sha256-mh9UdYhZl6JVJEeDFnY5BjfcK+PrWBBEn1Qh7/ZX17k=";
+  src =
+    if staticX86_64Musl
+    then
+      runCommand "nostr-bench-static-src" {} ''
+        cp -r ${srcBase} $out
+        chmod -R u+w $out
+        cp ${./nostr-bench-static.Cargo.lock} $out/Cargo.lock
+      ''
+    else srcBase;
+in
+  selectedRustPlatform.buildRustPackage (
+    {
+      pname = "nostr-bench";
+      version = "0.4.0";
 
-  doCheck = false;
+      inherit src;
 
-  meta = with lib; {
-    description = "Nostr relay benchmarking tool";
-    homepage = "https://github.com/rnostr/nostr-bench";
-    license = licenses.mit;
-    mainProgram = "nostr-bench";
-    platforms = platforms.unix;
-  };
-}
+      doCheck = false;
+
+      meta = with lib; {
+        description = "Nostr relay benchmarking tool";
+        homepage = "https://github.com/rnostr/nostr-bench";
+        license = licenses.mit;
+        mainProgram = "nostr-bench";
+        platforms = platforms.unix;
+      };
+    }
+    // lib.optionalAttrs staticX86_64Musl {
+      cargoHash = "sha256-aL8XSBJ8sHl7CGh9SkOoI+WlAHKrdij2DfvZAWIKgKY=";
+      CARGO_BUILD_TARGET = "x86_64-unknown-linux-musl";
+      RUSTFLAGS = "-C target-feature=+crt-static";
+    }
+    // lib.optionalAttrs (!staticX86_64Musl) {
+      cargoHash = "sha256-mh9UdYhZl6JVJEeDFnY5BjfcK+PrWBBEn1Qh7/ZX17k=";
+    }
+  )

package.json

@@ -1,5 +1,5 @@
 {
-  "dependencies": {
-    "@mariozechner/pi-coding-agent": "^0.57.1"
+  "devDependencies": {
+    "@mariozechner/pi-coding-agent": "^0.60.0"
   }
 }

scripts/cloud_bench_client.sh

@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+relay_url="${1:-}"
+mode="${2:-all}"
+
+if [[ -z "$relay_url" ]]; then
+  echo "usage: cloud-bench-client.sh <relay-url> [connect|echo|event|req|all]" >&2
+  exit 1
+fi
+
+bench_bin="${NOSTR_BENCH_BIN:-/usr/local/bin/nostr-bench}"
+bench_threads="${PARRHESIA_BENCH_THREADS:-0}"
+client_nofile="${PARRHESIA_BENCH_CLIENT_NOFILE:-262144}"
+
+ulimit -n "${client_nofile}" >/dev/null 2>&1 || true
+
+run_connect() {
+  echo "==> nostr-bench connect ${relay_url}"
+  "$bench_bin" connect --json \
+    -c "${PARRHESIA_BENCH_CONNECT_COUNT:-200}" \
+    -r "${PARRHESIA_BENCH_CONNECT_RATE:-100}" \
+    -k "${PARRHESIA_BENCH_KEEPALIVE_SECONDS:-5}" \
+    -t "${bench_threads}" \
+    "${relay_url}"
+}
+
+run_echo() {
+  echo "==> nostr-bench echo ${relay_url}"
+  "$bench_bin" echo --json \
+    -c "${PARRHESIA_BENCH_ECHO_COUNT:-100}" \
+    -r "${PARRHESIA_BENCH_ECHO_RATE:-50}" \
+    -k "${PARRHESIA_BENCH_KEEPALIVE_SECONDS:-5}" \
+    -t "${bench_threads}" \
+    --size "${PARRHESIA_BENCH_ECHO_SIZE:-512}" \
+    "${relay_url}"
+}
+
+run_event() {
+  echo "==> nostr-bench event ${relay_url}"
+  "$bench_bin" event --json \
+    -c "${PARRHESIA_BENCH_EVENT_COUNT:-100}" \
+    -r "${PARRHESIA_BENCH_EVENT_RATE:-50}" \
+    -k "${PARRHESIA_BENCH_KEEPALIVE_SECONDS:-5}" \
+    -t "${bench_threads}" \
+    "${relay_url}"
+}
+
+run_req() {
+  echo "==> nostr-bench req ${relay_url}"
+  "$bench_bin" req --json \
+    -c "${PARRHESIA_BENCH_REQ_COUNT:-100}" \
+    -r "${PARRHESIA_BENCH_REQ_RATE:-50}" \
+    -k "${PARRHESIA_BENCH_KEEPALIVE_SECONDS:-5}" \
+    -t "${bench_threads}" \
+    --limit "${PARRHESIA_BENCH_REQ_LIMIT:-10}" \
+    "${relay_url}"
+}
+
+case "$mode" in
+  connect) run_connect ;;
+  echo)    run_echo ;;
+  event)   run_event ;;
+  req)     run_req ;;
+  all)     run_connect; echo; run_echo; echo; run_event; echo; run_req ;;
+  *)       echo "unknown mode: $mode" >&2; exit 1 ;;
+esac

scripts/cloud_bench_monitoring.mjs

@@ -0,0 +1,148 @@
+// cloud_bench_monitoring.mjs — Prometheus + node_exporter setup and metrics collection.
+//
+// Installs monitoring on ephemeral benchmark VMs, collects all Prometheus
+// metrics for a given time window via the HTTP API, and stores them as
+// JSON artifacts.
+
+// Generate prometheus.yml scrape config.
+export function makePrometheusConfig({ clientIps = [] } = {}) {
+  const targets = [
+    {
+      job_name: "node-server",
+      static_configs: [{ targets: ["localhost:9100"] }],
+    },
+    {
+      job_name: "relay",
+      metrics_path: "/metrics",
+      static_configs: [{ targets: ["localhost:4413"] }],
+    },
+  ];
+
+  if (clientIps.length > 0) {
+    targets.push({
+      job_name: "node-clients",
+      static_configs: [{ targets: clientIps.map((ip) => `${ip}:9100`) }],
+    });
+  }
+
+  const config = {
+    global: {
+      scrape_interval: "5s",
+      evaluation_interval: "15s",
+    },
+    scrape_configs: targets,
+  };
+
+  // Produce minimal YAML by hand (avoids adding a yaml dep).
+  const lines = [
+    "global:",
+    "  scrape_interval: 5s",
+    "  evaluation_interval: 15s",
+    "",
+    "scrape_configs:",
+  ];
+
+  for (const sc of targets) {
+    lines.push(`  - job_name: '${sc.job_name}'`);
+    if (sc.metrics_path) {
+      lines.push(`    metrics_path: '${sc.metrics_path}'`);
+    }
+    lines.push("    static_configs:");
+    for (const st of sc.static_configs) {
+      lines.push("      - targets:");
+      for (const t of st.targets) {
+        lines.push(`          - '${t}'`);
+      }
+    }
+  }
+
+  return lines.join("\n") + "\n";
+}
+
+// Install Prometheus + node_exporter on server, node_exporter on clients.
+// `ssh` is an async function matching the sshExec(ip, keyPath, cmd, opts) signature.
+export async function installMonitoring({ serverIp, clientIps = [], keyPath, ssh }) {
+  const prometheusYml = makePrometheusConfig({ clientIps });
+
+  // Server: install prometheus + node-exporter, write config, start
+  console.log("[monitoring] installing prometheus + node-exporter on server");
+  await ssh(serverIp, keyPath, [
+    "export DEBIAN_FRONTEND=noninteractive",
+    "apt-get update -qq",
+    "apt-get install -y -qq prometheus prometheus-node-exporter >/dev/null 2>&1",
+  ].join(" && "));
+
+  // Write prometheus config
+  const escapedYml = prometheusYml.replace(/'/g, "'\\''");
+  await ssh(serverIp, keyPath, `cat > /etc/prometheus/prometheus.yml <<'PROMEOF'\n${prometheusYml}PROMEOF`);
+
+  // Restart prometheus with the new config, ensure node-exporter is running
+  await ssh(serverIp, keyPath, [
+    "systemctl restart prometheus",
+    "systemctl enable --now prometheus-node-exporter",
+  ].join(" && "));
+
+  // Clients: install node-exporter only (in parallel)
+  if (clientIps.length > 0) {
+    console.log(`[monitoring] installing node-exporter on ${clientIps.length} client(s)`);
+    await Promise.all(
+      clientIps.map((ip) =>
+        ssh(ip, keyPath, [
+          "export DEBIAN_FRONTEND=noninteractive",
+          "apt-get update -qq",
+          "apt-get install -y -qq prometheus-node-exporter >/dev/null 2>&1",
+          "systemctl enable --now prometheus-node-exporter",
+        ].join(" && "))
+      )
+    );
+  }
+
+  // Wait for Prometheus to start scraping
+  console.log("[monitoring] waiting for Prometheus to initialise");
+  await ssh(serverIp, keyPath,
+    'for i in $(seq 1 30); do curl -sf http://localhost:9090/api/v1/query?query=up >/dev/null 2>&1 && exit 0; sleep 1; done; echo "prometheus not ready" >&2; exit 1'
+  );
+
+  console.log("[monitoring] monitoring active");
+}
+
+// Collect all Prometheus metrics for a time window.
+// Returns the raw Prometheus API response JSON (matrix result type).
+export async function collectMetrics({ serverIp, startTime, endTime, step = 5 }) {
+  const params = new URLSearchParams({
+    query: '{__name__=~".+"}',
+    start: startTime,
+    end: endTime,
+    step: String(step),
+  });
+
+  const url = `http://${serverIp}:9090/api/v1/query_range?${params}`;
+
+  try {
+    const resp = await fetch(url, { signal: AbortSignal.timeout(60_000) });
+    if (!resp.ok) {
+      console.error(`[monitoring] Prometheus query failed: ${resp.status} ${resp.statusText}`);
+      return null;
+    }
+    const body = await resp.json();
+    if (body.status !== "success") {
+      console.error(`[monitoring] Prometheus query error: ${body.error || "unknown"}`);
+      return null;
+    }
+    return body.data;
+  } catch (err) {
+    console.error(`[monitoring] metrics collection failed: ${err.message}`);
+    return null;
+  }
+}
+
+// Stop monitoring daemons on server and clients.
+export async function stopMonitoring({ serverIp, clientIps = [], keyPath, ssh }) {
+  const allIps = [serverIp, ...clientIps];
+  await Promise.all(
+    allIps.map((ip) =>
+      ssh(ip, keyPath, "systemctl stop prometheus prometheus-node-exporter 2>/dev/null; true").catch(() => {})
+    )
+  );
+  console.log("[monitoring] monitoring stopped");
+}

scripts/cloud_bench_results.mjs

@@ -0,0 +1,223 @@
+// cloud_bench_results.mjs — benchmark output parsing and result aggregation.
+//
+// Extracted from cloud_bench_orchestrate.mjs to keep the orchestrator focused
+// on provisioning and execution flow.
+
+export function parseNostrBenchSections(output) {
+  const lines = output.split(/\r?\n/);
+  let section = null;
+  const parsed = {};
+
+  for (const lineRaw of lines) {
+    const line = lineRaw.trim();
+    const header = line.match(/^==>\s+nostr-bench\s+(connect|echo|event|req)\s+/);
+    if (header) {
+      section = header[1];
+      continue;
+    }
+
+    if (!line.startsWith("{")) continue;
+
+    try {
+      const json = JSON.parse(line);
+      if (section) {
+        parsed[section] = json;
+      }
+    } catch {
+      // ignore noisy non-json lines
+    }
+  }
+
+  return parsed;
+}
+
+export function mean(values) {
+  const valid = values.filter((v) => Number.isFinite(v));
+  if (valid.length === 0) return NaN;
+  return valid.reduce((a, b) => a + b, 0) / valid.length;
+}
+
+export function sum(values) {
+  const valid = values.filter((v) => Number.isFinite(v));
+  if (valid.length === 0) return NaN;
+  return valid.reduce((a, b) => a + b, 0);
+}
+
+export function throughputFromSection(section) {
+  const elapsedMs = Number(section?.elapsed ?? NaN);
+  const complete = Number(section?.message_stats?.complete ?? NaN);
+  const totalBytes = Number(section?.message_stats?.size ?? NaN);
+
+  const cumulativeTps =
+    Number.isFinite(elapsedMs) && elapsedMs > 0 && Number.isFinite(complete)
+      ? complete / (elapsedMs / 1000)
+      : NaN;
+
+  const cumulativeMibs =
+    Number.isFinite(elapsedMs) && elapsedMs > 0 && Number.isFinite(totalBytes)
+      ? totalBytes / (1024 * 1024) / (elapsedMs / 1000)
+      : NaN;
+
+  const sampleTps = Number(section?.tps ?? NaN);
+  const sampleMibs = Number(section?.size ?? NaN);
+
+  return {
+    tps: Number.isFinite(cumulativeTps) ? cumulativeTps : sampleTps,
+    mibs: Number.isFinite(cumulativeMibs) ? cumulativeMibs : sampleMibs,
+  };
+}
+
+export function metricFromSections(sections) {
+  const connect = sections?.connect?.connect_stats?.success_time || {};
+  const echo = throughputFromSection(sections?.echo || {});
+  const event = throughputFromSection(sections?.event || {});
+  const req = throughputFromSection(sections?.req || {});
+
+  return {
+    connect_avg_ms: Number(connect.avg ?? NaN),
+    connect_max_ms: Number(connect.max ?? NaN),
+    echo_tps: echo.tps,
+    echo_mibs: echo.mibs,
+    event_tps: event.tps,
+    event_mibs: event.mibs,
+    req_tps: req.tps,
+    req_mibs: req.mibs,
+  };
+}
+
+export function summariseFlatResults(results) {
+  const byServer = new Map();
+
+  for (const runEntry of results) {
+    const serverName = runEntry.target;
+    if (!byServer.has(serverName)) {
+      byServer.set(serverName, []);
+    }
+
+    const clientSamples = (runEntry.clients || [])
+      .filter((clientResult) => clientResult.status === "ok")
+      .map((clientResult) => metricFromSections(clientResult.sections || {}));
+
+    if (clientSamples.length === 0) {
+      continue;
+    }
+
+    byServer.get(serverName).push({
+      connect_avg_ms: mean(clientSamples.map((s) => s.connect_avg_ms)),
+      connect_max_ms: mean(clientSamples.map((s) => s.connect_max_ms)),
+      echo_tps: sum(clientSamples.map((s) => s.echo_tps)),
+      echo_mibs: sum(clientSamples.map((s) => s.echo_mibs)),
+      event_tps: sum(clientSamples.map((s) => s.event_tps)),
+      event_mibs: sum(clientSamples.map((s) => s.event_mibs)),
+      req_tps: sum(clientSamples.map((s) => s.req_tps)),
+      req_mibs: sum(clientSamples.map((s) => s.req_mibs)),
+    });
+  }
+
+  const metricKeys = [
+    "connect_avg_ms",
+    "connect_max_ms",
+    "echo_tps",
+    "echo_mibs",
+    "event_tps",
+    "event_mibs",
+    "req_tps",
+    "req_mibs",
+  ];
+
+  const out = {};
+  for (const [serverName, runSamples] of byServer.entries()) {
+    const summary = {};
+    for (const key of metricKeys) {
+      summary[key] = mean(runSamples.map((s) => s[key]));
+    }
+    out[serverName] = summary;
+  }
+
+  return out;
+}
+
+export function summarisePhasedResults(results) {
+  const byServer = new Map();
+
+  for (const entry of results) {
+    if (!byServer.has(entry.target)) byServer.set(entry.target, []);
+    const phases = entry.phases;
+    if (!phases) continue;
+
+    const sample = {};
+
+    // connect
+    const connectClients = (phases.connect?.clients || [])
+      .filter((c) => c.status === "ok")
+      .map((c) => metricFromSections(c.sections || {}));
+    if (connectClients.length > 0) {
+      sample.connect_avg_ms = mean(connectClients.map((s) => s.connect_avg_ms));
+      sample.connect_max_ms = mean(connectClients.map((s) => s.connect_max_ms));
+    }
+
+    // echo
+    const echoClients = (phases.echo?.clients || [])
+      .filter((c) => c.status === "ok")
+      .map((c) => metricFromSections(c.sections || {}));
+    if (echoClients.length > 0) {
+      sample.echo_tps = sum(echoClients.map((s) => s.echo_tps));
+      sample.echo_mibs = sum(echoClients.map((s) => s.echo_mibs));
+    }
+
+    // Per-level req and event metrics
+    for (const level of ["empty", "warm", "hot"]) {
+      const phase = phases[level];
+      if (!phase) continue;
+
+      const reqClients = (phase.req?.clients || [])
+        .filter((c) => c.status === "ok")
+        .map((c) => metricFromSections(c.sections || {}));
+      if (reqClients.length > 0) {
+        sample[`req_${level}_tps`] = sum(reqClients.map((s) => s.req_tps));
+        sample[`req_${level}_mibs`] = sum(reqClients.map((s) => s.req_mibs));
+      }
+
+      const eventClients = (phase.event?.clients || [])
+        .filter((c) => c.status === "ok")
+        .map((c) => metricFromSections(c.sections || {}));
+      if (eventClients.length > 0) {
+        sample[`event_${level}_tps`] = sum(eventClients.map((s) => s.event_tps));
+        sample[`event_${level}_mibs`] = sum(eventClients.map((s) => s.event_mibs));
+      }
+    }
+
+    byServer.get(entry.target).push(sample);
+  }
+
+  const out = {};
+  for (const [name, samples] of byServer.entries()) {
+    if (samples.length === 0) continue;
+    const allKeys = new Set(samples.flatMap((s) => Object.keys(s)));
+    const summary = {};
+    for (const key of allKeys) {
+      summary[key] = mean(samples.map((s) => s[key]).filter((v) => v !== undefined));
+    }
+    out[name] = summary;
+  }
+
+  return out;
+}
+
+export function summariseServersFromResults(results) {
+  const isPhased = results.some((r) => r.mode === "phased");
+  return isPhased ? summarisePhasedResults(results) : summariseFlatResults(results);
+}
+
+// Count events successfully written by event benchmarks across all clients.
+export function countEventsWritten(clientResults) {
+  let total = 0;
+  for (const cr of clientResults) {
+    if (cr.status !== "ok") continue;
+    const eventSection = cr.sections?.event;
+    if (eventSection?.message_stats?.complete) {
+      total += Number(eventSection.message_stats.complete) || 0;
+    }
+  }
+  return total;
+}

scripts/cloud_bench_server.sh

@@ -0,0 +1,637 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+PARRHESIA_IMAGE="${PARRHESIA_IMAGE:-parrhesia:latest}"
+POSTGRES_IMAGE="${POSTGRES_IMAGE:-postgres:18}"
+STRFRY_IMAGE="${STRFRY_IMAGE:-ghcr.io/hoytech/strfry:latest}"
+NOSTR_RS_IMAGE="${NOSTR_RS_IMAGE:-scsibug/nostr-rs-relay:latest}"
+NOSTREAM_REPO="${NOSTREAM_REPO:-https://github.com/Cameri/nostream.git}"
+NOSTREAM_REF="${NOSTREAM_REF:-main}"
+NOSTREAM_REDIS_IMAGE="${NOSTREAM_REDIS_IMAGE:-redis:7.0.5-alpine3.16}"
+HAVEN_IMAGE="${HAVEN_IMAGE:-holgerhatgarkeinenode/haven-docker:latest}"
+HAVEN_RELAY_URL="${HAVEN_RELAY_URL:-127.0.0.1:3355}"
+
+NOSTREAM_SECRET="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+HAVEN_OWNER_NPUB="npub1utx00neqgqln72j22kej3ux7803c2k986henvvha4thuwfkper4s7r50e8"
+
+cleanup_containers() {
+  docker rm -f parrhesia pg strfry nostr-rs nostream nostream-db nostream-cache haven >/dev/null 2>&1 || true
+}
+
+ensure_benchnet() {
+  docker network create benchnet >/dev/null 2>&1 || true
+}
+
+wait_http() {
+  local url="$1"
+  local timeout="${2:-60}"
+  local log_container="${3:-}"
+
+  for _ in $(seq 1 "$timeout"); do
+    if curl -fsS "$url" >/dev/null 2>&1; then
+      return 0
+    fi
+    sleep 1
+  done
+
+  if [[ -n "$log_container" ]]; then
+    docker logs --tail 200 "$log_container" >&2 || true
+  fi
+
+  echo "Timed out waiting for HTTP endpoint: $url" >&2
+  return 1
+}
+
+wait_pg() {
+  local timeout="${1:-90}"
+  for _ in $(seq 1 "$timeout"); do
+    if docker exec pg pg_isready -U parrhesia -d parrhesia >/dev/null 2>&1; then
+      return 0
+    fi
+    sleep 1
+  done
+  docker logs --tail 200 pg >&2 || true
+  echo "Timed out waiting for Postgres" >&2
+  return 1
+}
+
+wait_nostream_pg() {
+  local timeout="${1:-90}"
+  for _ in $(seq 1 "$timeout"); do
+    if docker exec nostream-db pg_isready -U nostr_ts_relay -d nostr_ts_relay >/dev/null 2>&1; then
+      return 0
+    fi
+    sleep 1
+  done
+  docker logs --tail 200 nostream-db >&2 || true
+  echo "Timed out waiting for nostream Postgres" >&2
+  return 1
+}
+
+wait_nostream_redis() {
+  local timeout="${1:-60}"
+  for _ in $(seq 1 "$timeout"); do
+    if docker exec nostream-cache redis-cli -a nostr_ts_relay ping >/dev/null 2>&1; then
+      return 0
+    fi
+    sleep 1
+  done
+  docker logs --tail 200 nostream-cache >&2 || true
+  echo "Timed out waiting for nostream Redis" >&2
+  return 1
+}
+
+wait_port() {
+  local port="$1"
+  local timeout="${2:-60}"
+  local log_container="${3:-}"
+
+  for _ in $(seq 1 "$timeout"); do
+    if ss -ltn | grep -q ":${port} "; then
+      return 0
+    fi
+    sleep 1
+  done
+
+  if [[ -n "$log_container" ]]; then
+    docker logs --tail 200 "$log_container" >&2 || true
+  fi
+
+  echo "Timed out waiting for port: $port" >&2
+  return 1
+}
+
+clamp() {
+  local value="$1"
+  local min="$2"
+  local max="$3"
+
+  if (( value < min )); then
+    echo "$min"
+  elif (( value > max )); then
+    echo "$max"
+  else
+    echo "$value"
+  fi
+}
+
+derive_resource_tuning() {
+  local mem_kb
+  mem_kb="$(awk '/MemTotal:/ {print $2}' /proc/meminfo 2>/dev/null || true)"
+
+  if [[ -z "$mem_kb" || ! "$mem_kb" =~ ^[0-9]+$ ]]; then
+    mem_kb=4194304
+  fi
+
+  HOST_MEM_MB=$((mem_kb / 1024))
+  HOST_CPU_CORES=$(nproc 2>/dev/null || echo 2)
+
+  local computed_pg_max_connections=$((HOST_CPU_CORES * 50))
+  local computed_pg_shared_buffers_mb=$((HOST_MEM_MB / 4))
+  local computed_pg_effective_cache_size_mb=$((HOST_MEM_MB * 3 / 4))
+  local computed_pg_maintenance_work_mem_mb=$((HOST_MEM_MB / 16))
+  local computed_pg_max_wal_size_gb=$((HOST_MEM_MB / 8192))
+
+  computed_pg_max_connections=$(clamp "$computed_pg_max_connections" 200 1000)
+  computed_pg_shared_buffers_mb=$(clamp "$computed_pg_shared_buffers_mb" 512 32768)
+  computed_pg_effective_cache_size_mb=$(clamp "$computed_pg_effective_cache_size_mb" 1024 98304)
+  computed_pg_maintenance_work_mem_mb=$(clamp "$computed_pg_maintenance_work_mem_mb" 256 2048)
+  computed_pg_max_wal_size_gb=$(clamp "$computed_pg_max_wal_size_gb" 4 64)
+
+  local computed_pg_min_wal_size_gb=$((computed_pg_max_wal_size_gb / 4))
+  computed_pg_min_wal_size_gb=$(clamp "$computed_pg_min_wal_size_gb" 1 16)
+
+  local computed_pg_work_mem_mb=$(((HOST_MEM_MB - computed_pg_shared_buffers_mb) / (computed_pg_max_connections * 3)))
+  computed_pg_work_mem_mb=$(clamp "$computed_pg_work_mem_mb" 4 128)
+
+  local computed_parrhesia_pool_size=$((HOST_CPU_CORES * 8))
+  computed_parrhesia_pool_size=$(clamp "$computed_parrhesia_pool_size" 20 200)
+
+  local computed_nostream_db_min_pool_size=$((HOST_CPU_CORES * 4))
+  computed_nostream_db_min_pool_size=$(clamp "$computed_nostream_db_min_pool_size" 16 128)
+
+  local computed_nostream_db_max_pool_size=$((HOST_CPU_CORES * 16))
+  computed_nostream_db_max_pool_size=$(clamp "$computed_nostream_db_max_pool_size" 64 512)
+
+  if (( computed_nostream_db_max_pool_size < computed_nostream_db_min_pool_size )); then
+    computed_nostream_db_max_pool_size="$computed_nostream_db_min_pool_size"
+  fi
+
+  local computed_redis_maxmemory_mb=$((HOST_MEM_MB / 3))
+  computed_redis_maxmemory_mb=$(clamp "$computed_redis_maxmemory_mb" 256 65536)
+
+  PG_MAX_CONNECTIONS="${PG_MAX_CONNECTIONS:-$computed_pg_max_connections}"
+  PG_SHARED_BUFFERS_MB="${PG_SHARED_BUFFERS_MB:-$computed_pg_shared_buffers_mb}"
+  PG_EFFECTIVE_CACHE_SIZE_MB="${PG_EFFECTIVE_CACHE_SIZE_MB:-$computed_pg_effective_cache_size_mb}"
+  PG_MAINTENANCE_WORK_MEM_MB="${PG_MAINTENANCE_WORK_MEM_MB:-$computed_pg_maintenance_work_mem_mb}"
+  PG_WORK_MEM_MB="${PG_WORK_MEM_MB:-$computed_pg_work_mem_mb}"
+  PG_MIN_WAL_SIZE_GB="${PG_MIN_WAL_SIZE_GB:-$computed_pg_min_wal_size_gb}"
+  PG_MAX_WAL_SIZE_GB="${PG_MAX_WAL_SIZE_GB:-$computed_pg_max_wal_size_gb}"
+  PARRHESIA_POOL_SIZE="${PARRHESIA_POOL_SIZE:-$computed_parrhesia_pool_size}"
+  NOSTREAM_DB_MIN_POOL_SIZE="${NOSTREAM_DB_MIN_POOL_SIZE:-$computed_nostream_db_min_pool_size}"
+  NOSTREAM_DB_MAX_POOL_SIZE="${NOSTREAM_DB_MAX_POOL_SIZE:-$computed_nostream_db_max_pool_size}"
+  REDIS_MAXMEMORY_MB="${REDIS_MAXMEMORY_MB:-$computed_redis_maxmemory_mb}"
+
+  PG_TUNING_ARGS=(
+    -c max_connections="$PG_MAX_CONNECTIONS"
+    -c shared_buffers="${PG_SHARED_BUFFERS_MB}MB"
+    -c effective_cache_size="${PG_EFFECTIVE_CACHE_SIZE_MB}MB"
+    -c maintenance_work_mem="${PG_MAINTENANCE_WORK_MEM_MB}MB"
+    -c work_mem="${PG_WORK_MEM_MB}MB"
+    -c min_wal_size="${PG_MIN_WAL_SIZE_GB}GB"
+    -c max_wal_size="${PG_MAX_WAL_SIZE_GB}GB"
+    -c checkpoint_completion_target=0.9
+    -c wal_compression=on
+  )
+
+  echo "[server] resource profile: mem_mb=$HOST_MEM_MB cpu_cores=$HOST_CPU_CORES"
+  echo "[server] postgres tuning: max_connections=$PG_MAX_CONNECTIONS shared_buffers=${PG_SHARED_BUFFERS_MB}MB effective_cache_size=${PG_EFFECTIVE_CACHE_SIZE_MB}MB work_mem=${PG_WORK_MEM_MB}MB"
+  echo "[server] app tuning: parrhesia_pool=$PARRHESIA_POOL_SIZE nostream_db_pool=${NOSTREAM_DB_MIN_POOL_SIZE}-${NOSTREAM_DB_MAX_POOL_SIZE} redis_maxmemory=${REDIS_MAXMEMORY_MB}MB"
+}
+
+tune_nostream_settings() {
+  local settings_path="/root/nostream-config/settings.yaml"
+
+  if [[ ! -f "$settings_path" ]]; then
+    return 1
+  fi
+
+  python3 - "$settings_path" <<'PY'
+import pathlib
+import sys
+
+path = pathlib.Path(sys.argv[1])
+text = path.read_text(encoding="utf-8")
+
+def replace_after(marker: str, old: str, new: str) -> None:
+    global text
+    marker_idx = text.find(marker)
+    if marker_idx == -1:
+        return
+
+    old_idx = text.find(old, marker_idx)
+    if old_idx == -1:
+        return
+
+    text = text[:old_idx] + new + text[old_idx + len(old):]
+
+text = text.replace("  remoteIpHeader: x-forwarded-for", "  # remoteIpHeader disabled for direct bench traffic")
+
+text = text.replace(
+    "  connection:\\n    rateLimits:\\n      - period: 1000\\n        rate: 12\\n      - period: 60000\\n        rate: 48",
+    "  connection:\\n    rateLimits:\\n      - period: 1000\\n        rate: 300\\n      - period: 60000\\n        rate: 12000",
+)
+
+replace_after("description: 30 admission checks/min or 1 check every 2 seconds", "rate: 30", "rate: 3000")
+replace_after("description: 6 events/min for event kinds 0, 3, 40 and 41", "rate: 6", "rate: 600")
+replace_after("description: 12 events/min for event kinds 1, 2, 4 and 42", "rate: 12", "rate: 1200")
+replace_after("description: 30 events/min for event kind ranges 5-7 and 43-49", "rate: 30", "rate: 3000")
+replace_after("description: 24 events/min for replaceable events and parameterized replaceable", "rate: 24", "rate: 2400")
+replace_after("description: 60 events/min for ephemeral events", "rate: 60", "rate: 6000")
+replace_after("description: 720 events/hour for all events", "rate: 720", "rate: 72000")
+replace_after("description: 240 raw messages/min", "rate: 240", "rate: 120000")
+
+text = text.replace("maxSubscriptions: 10", "maxSubscriptions: 512")
+text = text.replace("maxFilters: 10", "maxFilters: 128")
+text = text.replace("maxFilterValues: 2500", "maxFilterValues: 100000")
+text = text.replace("maxLimit: 5000", "maxLimit: 50000")
+
+path.write_text(text, encoding="utf-8")
+PY
+}
+
+common_parrhesia_env=()
+common_parrhesia_env+=( -e PARRHESIA_ENABLE_EXPIRATION_WORKER=0 )
+common_parrhesia_env+=( -e PARRHESIA_ENABLE_PARTITION_RETENTION_WORKER=0 )
+common_parrhesia_env+=( -e PARRHESIA_PUBLIC_MAX_CONNECTIONS=infinity )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_FRAME_BYTES=16777216 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_EVENT_BYTES=4194304 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_FILTERS_PER_REQ=1024 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_FILTER_LIMIT=100000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_TAGS_PER_EVENT=4096 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_TAG_VALUES_PER_FILTER=4096 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_IP_MAX_EVENT_INGEST_PER_WINDOW=1000000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_RELAY_MAX_EVENT_INGEST_PER_WINDOW=1000000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_SUBSCRIPTIONS_PER_CONNECTION=4096 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_EVENT_FUTURE_SKEW_SECONDS=31536000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_EVENT_INGEST_PER_WINDOW=1000000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_AUTH_MAX_AGE_SECONDS=31536000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_OUTBOUND_QUEUE=65536 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_OUTBOUND_DRAIN_BATCH_SIZE=4096 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_NEGENTROPY_PAYLOAD_BYTES=1048576 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_NEGENTROPY_SESSIONS_PER_CONNECTION=256 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_NEGENTROPY_TOTAL_SESSIONS=100000 )
+common_parrhesia_env+=( -e PARRHESIA_LIMITS_MAX_NEGENTROPY_ITEMS_PER_SESSION=1000000 )
+
+cmd="${1:-}"
+if [[ -z "$cmd" ]]; then
+  echo "usage: cloud-bench-server.sh <start-*|wipe-data-*|cleanup>" >&2
+  exit 1
+fi
+
+derive_resource_tuning
+
+case "$cmd" in
+  start-parrhesia-pg)
+    cleanup_containers
+    docker network create benchnet >/dev/null 2>&1 || true
+
+    docker run -d --name pg --network benchnet \
+      --ulimit nofile=262144:262144 \
+      -e POSTGRES_DB=parrhesia \
+      -e POSTGRES_USER=parrhesia \
+      -e POSTGRES_PASSWORD=parrhesia \
+      "$POSTGRES_IMAGE" \
+      "${PG_TUNING_ARGS[@]}" >/dev/null
+
+    wait_pg 90
+
+    docker run --rm --network benchnet \
+      -e DATABASE_URL=ecto://parrhesia:parrhesia@pg:5432/parrhesia \
+      "$PARRHESIA_IMAGE" \
+      eval "Parrhesia.Release.migrate()"
+
+    docker run -d --name parrhesia --network benchnet \
+      --ulimit nofile=262144:262144 \
+      -p 4413:4413 \
+      -e DATABASE_URL=ecto://parrhesia:parrhesia@pg:5432/parrhesia \
+      -e POOL_SIZE="$PARRHESIA_POOL_SIZE" \
+      "${common_parrhesia_env[@]}" \
+      "$PARRHESIA_IMAGE" >/dev/null
+
+    wait_http "http://127.0.0.1:4413/health" 120 parrhesia
+    ;;
+
+  start-parrhesia-memory)
+    cleanup_containers
+
+    docker run -d --name parrhesia \
+      --ulimit nofile=262144:262144 \
+      -p 4413:4413 \
+      -e PARRHESIA_STORAGE_BACKEND=memory \
+      -e PARRHESIA_MODERATION_CACHE_ENABLED=0 \
+      "${common_parrhesia_env[@]}" \
+      "$PARRHESIA_IMAGE" >/dev/null
+
+    wait_http "http://127.0.0.1:4413/health" 120 parrhesia
+    ;;
+
+  start-strfry)
+    cleanup_containers
+
+    rm -rf /root/strfry-data
+    mkdir -p /root/strfry-data/strfry
+    cat > /root/strfry.conf <<'EOF'
+# generated by cloud bench script
+db = "/data/strfry"
+relay {
+  bind = "0.0.0.0"
+  port = 7777
+  nofiles = 131072
+}
+EOF
+
+    docker run -d --name strfry \
+      --ulimit nofile=262144:262144 \
+      -p 7777:7777 \
+      -v /root/strfry.conf:/etc/strfry.conf:ro \
+      -v /root/strfry-data:/data \
+      "$STRFRY_IMAGE" \
+      --config /etc/strfry.conf relay >/dev/null
+
+    wait_port 7777 60 strfry
+    ;;
+
+  start-nostr-rs-relay)
+    cleanup_containers
+
+    cat > /root/nostr-rs.toml <<'EOF'
+[database]
+engine = "sqlite"
+
+[network]
+address = "0.0.0.0"
+port = 8080
+ping_interval = 120
+
+[options]
+reject_future_seconds = 1800
+
+[limits]
+messages_per_sec = 5000
+subscriptions_per_min = 6000
+max_event_bytes = 1048576
+max_ws_message_bytes = 16777216
+max_ws_frame_bytes = 16777216
+broadcast_buffer = 65536
+event_persist_buffer = 16384
+limit_scrapers = false
+EOF
+
+    docker run -d --name nostr-rs \
+      --ulimit nofile=262144:262144 \
+      -p 8080:8080 \
+      -v /root/nostr-rs.toml:/usr/src/app/config.toml:ro \
+      "$NOSTR_RS_IMAGE" >/dev/null
+
+    wait_http "http://127.0.0.1:8080/" 60 nostr-rs
+    ;;
+
+  start-nostream)
+    cleanup_containers
+    ensure_benchnet
+
+    if [[ ! -d /root/nostream-src/.git ]]; then
+      git clone --depth 1 "$NOSTREAM_REPO" /root/nostream-src >/dev/null
+    fi
+
+    git -C /root/nostream-src fetch --depth 1 origin "$NOSTREAM_REF" >/dev/null 2>&1 || true
+    if git -C /root/nostream-src rev-parse --verify FETCH_HEAD >/dev/null 2>&1; then
+      git -C /root/nostream-src checkout --force FETCH_HEAD >/dev/null
+    else
+      git -C /root/nostream-src checkout --force "$NOSTREAM_REF" >/dev/null
+    fi
+
+    nostream_ref_marker=/root/nostream-src/.bench_ref
+    should_build_nostream=0
+    if ! docker image inspect nostream:bench >/dev/null 2>&1; then
+      should_build_nostream=1
+    elif [[ ! -f "$nostream_ref_marker" ]] || [[ "$(cat "$nostream_ref_marker")" != "$NOSTREAM_REF" ]]; then
+      should_build_nostream=1
+    fi
+
+    if [[ "$should_build_nostream" == "1" ]]; then
+      docker build -t nostream:bench /root/nostream-src >/dev/null
+      printf '%s\n' "$NOSTREAM_REF" > "$nostream_ref_marker"
+    fi
+
+    mkdir -p /root/nostream-config
+    if [[ ! -f /root/nostream-config/settings.yaml ]]; then
+      cp /root/nostream-src/resources/default-settings.yaml /root/nostream-config/settings.yaml
+    fi
+
+    tune_nostream_settings
+
+    docker run -d --name nostream-db --network benchnet \
+      --ulimit nofile=262144:262144 \
+      -e POSTGRES_DB=nostr_ts_relay \
+      -e POSTGRES_USER=nostr_ts_relay \
+      -e POSTGRES_PASSWORD=nostr_ts_relay \
+      "$POSTGRES_IMAGE" \
+      "${PG_TUNING_ARGS[@]}" >/dev/null
+
+    wait_nostream_pg 90
+
+    docker run -d --name nostream-cache --network benchnet \
+      "$NOSTREAM_REDIS_IMAGE" \
+      redis-server \
+      --loglevel warning \
+      --requirepass nostr_ts_relay \
+      --maxmemory "${REDIS_MAXMEMORY_MB}mb" \
+      --maxmemory-policy noeviction >/dev/null
+
+    wait_nostream_redis 60
+
+    docker run --rm --network benchnet \
+      -e DB_HOST=nostream-db \
+      -e DB_PORT=5432 \
+      -e DB_USER=nostr_ts_relay \
+      -e DB_PASSWORD=nostr_ts_relay \
+      -e DB_NAME=nostr_ts_relay \
+      -v /root/nostream-src/migrations:/code/migrations:ro \
+      -v /root/nostream-src/knexfile.js:/code/knexfile.js:ro \
+      node:18-alpine3.16 \
+      sh -lc 'cd /code && npm install --no-save --quiet knex@2.4.0 pg@8.8.0 && npx knex migrate:latest'
+
+    docker run -d --name nostream --network benchnet \
+      --ulimit nofile=262144:262144 \
+      -p 8008:8008 \
+      -e SECRET="$NOSTREAM_SECRET" \
+      -e RELAY_PORT=8008 \
+      -e NOSTR_CONFIG_DIR=/home/node/.nostr \
+      -e DB_HOST=nostream-db \
+      -e DB_PORT=5432 \
+      -e DB_USER=nostr_ts_relay \
+      -e DB_PASSWORD=nostr_ts_relay \
+      -e DB_NAME=nostr_ts_relay \
+      -e DB_MIN_POOL_SIZE="$NOSTREAM_DB_MIN_POOL_SIZE" \
+      -e DB_MAX_POOL_SIZE="$NOSTREAM_DB_MAX_POOL_SIZE" \
+      -e DB_ACQUIRE_CONNECTION_TIMEOUT=60000 \
+      -e REDIS_HOST=nostream-cache \
+      -e REDIS_PORT=6379 \
+      -e REDIS_USER=default \
+      -e REDIS_PASSWORD=nostr_ts_relay \
+      -v /root/nostream-config:/home/node/.nostr:ro \
+      nostream:bench >/dev/null
+
+    wait_port 8008 180 nostream
+    ;;
+
+  start-haven)
+    cleanup_containers
+
+    rm -rf /root/haven-bench
+    mkdir -p /root/haven-bench/db
+    mkdir -p /root/haven-bench/blossom
+    mkdir -p /root/haven-bench/templates/static
+
+    if [[ ! -f /root/haven-bench/templates/index.html ]]; then
+      cat > /root/haven-bench/templates/index.html <<'EOF'
+<!doctype html>
+<html>
+  <head>
+    <meta charset="utf-8" />
+    <title>Haven</title>
+  </head>
+  <body>
+    <h1>Haven</h1>
+  </body>
+</html>
+EOF
+    fi
+
+    printf '[]\n' > /root/haven-bench/relays_import.json
+    printf '[]\n' > /root/haven-bench/relays_blastr.json
+    printf '[]\n' > /root/haven-bench/blacklisted_npubs.json
+    printf '[]\n' > /root/haven-bench/whitelisted_npubs.json
+
+    cat > /root/haven-bench/haven.env <<EOF
+OWNER_NPUB=$HAVEN_OWNER_NPUB
+RELAY_URL=$HAVEN_RELAY_URL
+RELAY_PORT=3355
+RELAY_BIND_ADDRESS=0.0.0.0
+DB_ENGINE=badger
+LMDB_MAPSIZE=0
+BLOSSOM_PATH=blossom/
+PRIVATE_RELAY_NAME=Private Relay
+PRIVATE_RELAY_NPUB=$HAVEN_OWNER_NPUB
+PRIVATE_RELAY_DESCRIPTION=Private relay for benchmarking
+PRIVATE_RELAY_ICON=https://example.com/icon.png
+PRIVATE_RELAY_EVENT_IP_LIMITER_TOKENS_PER_INTERVAL=1000
+PRIVATE_RELAY_EVENT_IP_LIMITER_INTERVAL=1
+PRIVATE_RELAY_EVENT_IP_LIMITER_MAX_TOKENS=5000
+PRIVATE_RELAY_ALLOW_EMPTY_FILTERS=true
+PRIVATE_RELAY_ALLOW_COMPLEX_FILTERS=true
+PRIVATE_RELAY_CONNECTION_RATE_LIMITER_TOKENS_PER_INTERVAL=500
+PRIVATE_RELAY_CONNECTION_RATE_LIMITER_INTERVAL=1
+PRIVATE_RELAY_CONNECTION_RATE_LIMITER_MAX_TOKENS=2000
+CHAT_RELAY_NAME=Chat Relay
+CHAT_RELAY_NPUB=$HAVEN_OWNER_NPUB
+CHAT_RELAY_DESCRIPTION=Chat relay for benchmarking
+CHAT_RELAY_ICON=https://example.com/icon.png
+CHAT_RELAY_EVENT_IP_LIMITER_TOKENS_PER_INTERVAL=1000
+CHAT_RELAY_EVENT_IP_LIMITER_INTERVAL=1
+CHAT_RELAY_EVENT_IP_LIMITER_MAX_TOKENS=5000
+CHAT_RELAY_ALLOW_EMPTY_FILTERS=true
+CHAT_RELAY_ALLOW_COMPLEX_FILTERS=true
+CHAT_RELAY_CONNECTION_RATE_LIMITER_TOKENS_PER_INTERVAL=500
+CHAT_RELAY_CONNECTION_RATE_LIMITER_INTERVAL=1
+CHAT_RELAY_CONNECTION_RATE_LIMITER_MAX_TOKENS=2000
+OUTBOX_RELAY_NAME=Outbox Relay
+OUTBOX_RELAY_NPUB=$HAVEN_OWNER_NPUB
+OUTBOX_RELAY_DESCRIPTION=Outbox relay for benchmarking
+OUTBOX_RELAY_ICON=https://example.com/icon.png
+OUTBOX_RELAY_EVENT_IP_LIMITER_TOKENS_PER_INTERVAL=1000
+OUTBOX_RELAY_EVENT_IP_LIMITER_INTERVAL=1
+OUTBOX_RELAY_EVENT_IP_LIMITER_MAX_TOKENS=5000
+OUTBOX_RELAY_ALLOW_EMPTY_FILTERS=true
+OUTBOX_RELAY_ALLOW_COMPLEX_FILTERS=true
+OUTBOX_RELAY_CONNECTION_RATE_LIMITER_TOKENS_PER_INTERVAL=500
+OUTBOX_RELAY_CONNECTION_RATE_LIMITER_INTERVAL=1
+OUTBOX_RELAY_CONNECTION_RATE_LIMITER_MAX_TOKENS=2000
+INBOX_RELAY_NAME=Inbox Relay
+INBOX_RELAY_NPUB=$HAVEN_OWNER_NPUB
+INBOX_RELAY_DESCRIPTION=Inbox relay for benchmarking
+INBOX_RELAY_ICON=https://example.com/icon.png
+INBOX_RELAY_EVENT_IP_LIMITER_TOKENS_PER_INTERVAL=1000
+INBOX_RELAY_EVENT_IP_LIMITER_INTERVAL=1
+INBOX_RELAY_EVENT_IP_LIMITER_MAX_TOKENS=5000
+INBOX_RELAY_ALLOW_EMPTY_FILTERS=true
+INBOX_RELAY_ALLOW_COMPLEX_FILTERS=true
+INBOX_RELAY_CONNECTION_RATE_LIMITER_TOKENS_PER_INTERVAL=500
+INBOX_RELAY_CONNECTION_RATE_LIMITER_INTERVAL=1
+INBOX_RELAY_CONNECTION_RATE_LIMITER_MAX_TOKENS=2000
+INBOX_PULL_INTERVAL_SECONDS=600
+IMPORT_START_DATE=2023-01-20
+IMPORT_OWNER_NOTES_FETCH_TIMEOUT_SECONDS=60
+IMPORT_TAGGED_NOTES_FETCH_TIMEOUT_SECONDS=120
+IMPORT_SEED_RELAYS_FILE=/app/relays_import.json
+BACKUP_PROVIDER=none
+BACKUP_INTERVAL_HOURS=24
+BLASTR_RELAYS_FILE=/app/relays_blastr.json
+BLASTR_TIMEOUT_SECONDS=5
+WOT_DEPTH=3
+WOT_MINIMUM_FOLLOWERS=0
+WOT_FETCH_TIMEOUT_SECONDS=30
+WOT_REFRESH_INTERVAL=24h
+WHITELISTED_NPUBS_FILE=
+BLACKLISTED_NPUBS_FILE=
+HAVEN_LOG_LEVEL=INFO
+EOF
+
+    chmod -R a+rwX /root/haven-bench
+
+    docker run -d --name haven \
+      --ulimit nofile=262144:262144 \
+      -p 3355:3355 \
+      --env-file /root/haven-bench/haven.env \
+      -v /root/haven-bench/db:/app/db \
+      -v /root/haven-bench/blossom:/app/blossom \
+      -v /root/haven-bench/templates:/app/templates \
+      -v /root/haven-bench/relays_import.json:/app/relays_import.json \
+      -v /root/haven-bench/relays_blastr.json:/app/relays_blastr.json \
+      -v /root/haven-bench/blacklisted_npubs.json:/app/blacklisted_npubs.json \
+      -v /root/haven-bench/whitelisted_npubs.json:/app/whitelisted_npubs.json \
+      "$HAVEN_IMAGE" >/dev/null
+
+    wait_port 3355 120 haven
+    ;;
+
+  wipe-data-parrhesia-pg)
+    docker exec pg psql -U parrhesia -d parrhesia -c \
+      "TRUNCATE event_ids, event_tags, events, replaceable_event_state, addressable_event_state CASCADE"
+    ;;
+
+  wipe-data-parrhesia-memory)
+    docker restart parrhesia
+    wait_http "http://127.0.0.1:4413/health" 120 parrhesia
+    ;;
+
+  wipe-data-strfry)
+    docker stop strfry
+    rm -rf /root/strfry-data/strfry/*
+    docker start strfry
+    wait_port 7777 60 strfry
+    ;;
+
+  wipe-data-nostr-rs-relay)
+    docker rm -f nostr-rs
+    docker run -d --name nostr-rs \
+      --ulimit nofile=262144:262144 \
+      -p 8080:8080 \
+      -v /root/nostr-rs.toml:/usr/src/app/config.toml:ro \
+      "$NOSTR_RS_IMAGE" >/dev/null
+    wait_http "http://127.0.0.1:8080/" 60 nostr-rs
+    ;;
+
+  wipe-data-nostream)
+    docker exec nostream-db psql -U nostr_ts_relay -d nostr_ts_relay -c \
+      "TRUNCATE events CASCADE"
+    ;;
+
+  wipe-data-haven)
+    docker stop haven
+    rm -rf /root/haven-bench/db/*
+    docker start haven
+    wait_port 3355 120 haven
+    ;;
+
+  cleanup)
+    cleanup_containers
+    ;;
+
+  *)
+    echo "unknown command: $cmd" >&2
+    exit 1
+    ;;
+esac

scripts/just_help.sh

@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+topic="${1:-}"
+
+if [[ -z "$topic" || "$topic" == "root" || "$topic" == "all" ]]; then
+  cat <<'EOF'
+Parrhesia command runner
+
+Usage:
+  just <group> <subcommand> [args...]
+  just help [group]
+
+Command groups:
+  e2e <subcommand>              End-to-end harness entrypoints
+  bench <subcommand>            Benchmark tasks (local/cloud/history/chart)
+
+Notes:
+  - Keep using mix aliases for core project workflows:
+      mix setup
+      mix test
+      mix lint
+      mix precommit
+
+Examples:
+  just help bench
+  just e2e marmot
+  just e2e node-sync
+  just bench compare
+  just bench cloud --clients 3 --runs 3
+EOF
+  exit 0
+fi
+
+if [[ "$topic" == "e2e" ]]; then
+  cat <<'EOF'
+E2E commands
+
+  just e2e nak                  CLI e2e tests via nak
+  just e2e marmot               Marmot client e2e tests
+  just e2e node-sync            Local two-node sync harness
+  just e2e node-sync-docker     Docker two-node sync harness
+
+Advanced:
+  just e2e suite <name> <cmd...>
+    -> runs scripts/run_e2e_suite.sh directly
+EOF
+  exit 0
+fi
+
+if [[ "$topic" == "bench" ]]; then
+  cat <<'EOF'
+Benchmark commands
+
+  just bench compare            Local benchmark comparison table only
+  just bench collect            Append run results to bench/history.jsonl
+  just bench update             Regenerate chart + README table
+  just bench list               List machines/runs from history
+  just bench at <git-ref>       Run collect benchmark at git ref
+  just bench cloud [args...]    Cloud benchmark wrapper
+  just bench cloud-quick        Cloud smoke profile
+
+Cloud tip:
+  just bench cloud --yes --datacenter auto
+    -> auto-pick cheapest compatible DC and skip interactive confirmation
+
+Cloud defaults:
+  targets = parrhesia-pg,parrhesia-memory,strfry,nostr-rs-relay,nostream,haven
+
+Relay-target helpers:
+  just bench relay [all|connect|echo|event|req] [nostr-bench-args...]
+  just bench relay-strfry [...]
+  just bench relay-nostr-rs [...]
+
+Examples:
+  just bench compare
+  just bench collect
+  just bench update --machine all
+  just bench at v0.5.0
+  just bench cloud --clients 3 --runs 3
+  just bench cloud --targets parrhesia-pg,nostream,haven --nostream-ref main
+EOF
+  exit 0
+fi
+
+echo "Unknown help topic: $topic" >&2
+echo "Run: just help" >&2
+exit 1

scripts/node_sync_e2e.exs

@@ -2,12 +2,14 @@ defmodule NodeSyncE2E.RelayClient do
   use WebSockex
 
   def start_link(url, owner, opts \\ []) do
-    WebSockex.start_link(
-      url,
-      __MODULE__,
-      owner,
-      Keyword.put(opts, :handle_initial_conn_failure, true)
-    )
+    ws_opts =
+      opts
+      |> Keyword.put_new(:handle_initial_conn_failure, true)
+      |> Keyword.put_new(:async, true)
+      |> Keyword.put_new(:socket_connect_timeout, 2_000)
+      |> Keyword.put_new(:socket_recv_timeout, 2_000)
+
+    WebSockex.start_link(url, __MODULE__, owner, ws_opts)
   end
 
   def send_json(pid, payload) do
@@ -163,6 +165,84 @@ defmodule NodeSyncE2E.Runner do
     end
   end
 
+  defp dispatch("filter-selectivity", config, opts) do
+    with {:ok, state_file} <- fetch_state_file(opts),
+         state = load_state(state_file),
+         :ok <- ensure_run_matches(config, state),
+         :ok <- ensure_nodes_ready(config),
+         :ok <- wait_for_sync_connected(config, config.node_b, config.server_id),
+         {:ok, non_matching_event} <- publish_non_matching_event(config, config.node_a),
+         _ = Process.sleep(2_000),
+         :ok <- ensure_event_absent(config, config.node_b, non_matching_event["id"]),
+         :ok <-
+           save_state(state_file, %{
+             "run_id" => config.run_id,
+             "resource" => config.resource,
+             "server_id" => config.server_id,
+             "non_matching_event_id" => non_matching_event["id"]
+           }) do
+      :ok
+    end
+  end
+
+  defp dispatch("sync-stop-restart", config, opts) do
+    with {:ok, state_file} <- fetch_state_file(opts),
+         state = load_state(state_file),
+         :ok <- ensure_run_matches(config, state),
+         :ok <- ensure_nodes_ready(config),
+         {:ok, %{"ok" => true}} <-
+           management_call(config, config.node_b, "sync_stop_server", %{"id" => config.server_id}),
+         {:ok, while_stopped_event} <-
+           publish_phase_event(config, config.node_a, "while-stopped"),
+         _ = Process.sleep(2_000),
+         :ok <- ensure_event_absent(config, config.node_b, while_stopped_event["id"]),
+         {:ok, %{"ok" => true}} <-
+           management_call(config, config.node_b, "sync_start_server", %{
+             "id" => config.server_id
+           }),
+         :ok <- wait_for_sync_connected(config, config.node_b, config.server_id),
+         :ok <- wait_for_event(config, config.node_b, while_stopped_event["id"]),
+         :ok <-
+           save_state(state_file, %{
+             "run_id" => config.run_id,
+             "resource" => config.resource,
+             "server_id" => config.server_id,
+             "while_stopped_event_id" => while_stopped_event["id"]
+           }) do
+      :ok
+    end
+  end
+
+  defp dispatch("bidirectional-sync", config, opts) do
+    with {:ok, state_file} <- fetch_state_file(opts),
+         state = load_state(state_file),
+         :ok <- ensure_run_matches(config, state),
+         {:ok, node_a_pubkey} <- fetch_state_value(state, "node_a_pubkey"),
+         {:ok, node_b_pubkey} <- fetch_state_value(state, "node_b_pubkey"),
+         :ok <- ensure_nodes_ready(config),
+         :ok <- ensure_acl(config, config.node_b, node_a_pubkey, "sync_read", config.filter),
+         :ok <-
+           ensure_acl(config, config.node_b, config.client_pubkey, "sync_write", config.filter),
+         :ok <- ensure_acl(config, config.node_a, node_b_pubkey, "sync_write", config.filter),
+         :ok <-
+           ensure_acl(config, config.node_a, config.client_pubkey, "sync_read", config.filter),
+         reverse_server_id = "node-b-upstream",
+         :ok <- configure_reverse_sync(config, node_b_pubkey, reverse_server_id),
+         :ok <- wait_for_sync_connected(config, config.node_a, reverse_server_id),
+         {:ok, bidir_event} <- publish_phase_event(config, config.node_b, "bidirectional"),
+         :ok <- wait_for_event(config, config.node_a, bidir_event["id"]),
+         :ok <-
+           save_state(state_file, %{
+             "run_id" => config.run_id,
+             "resource" => config.resource,
+             "server_id" => config.server_id,
+             "reverse_server_id" => reverse_server_id,
+             "bidir_event_id" => bidir_event["id"]
+           }) do
+      :ok
+    end
+  end
+
   defp dispatch(other, _config, _opts), do: {:error, {:unknown_command, other}}
 
   defp fetch_state_file(opts) do
@@ -334,6 +414,77 @@ defmodule NodeSyncE2E.Runner do
     %{"mode" => "disabled", "pins" => []}
   end
 
+  defp configure_reverse_sync(config, node_b_pubkey, reverse_server_id) do
+    params = %{
+      "id" => reverse_server_id,
+      "url" => config.node_b.sync_url,
+      "enabled?" => true,
+      "auth_pubkey" => node_b_pubkey,
+      "filters" => [config.filter],
+      "tls" => sync_tls_config(config.node_b.sync_url)
+    }
+
+    with {:ok, _server} <- management_call(config, config.node_a, "sync_put_server", params),
+         {:ok, %{"ok" => true}} <-
+           management_call(config, config.node_a, "sync_start_server", %{
+             "id" => reverse_server_id
+           }) do
+      :ok
+    end
+  end
+
+  defp publish_non_matching_event(config, node) do
+    event =
+      %{
+        "created_at" => System.system_time(:second),
+        "kind" => 5001,
+        "tags" => [
+          ["r", config.resource],
+          ["t", @subsystem_tag],
+          ["run", config.run_id],
+          ["phase", "filter-selectivity"]
+        ],
+        "content" => "filter-selectivity:#{config.run_id}"
+      }
+      |> sign_event!(config.client_private_key)
+
+    with {:ok, client} <- RelayClient.start_link(node.websocket_url, self()),
+         :ok <- await_client_connect(client) do
+      try do
+        case publish_event(client, node.relay_auth_url, config.client_private_key, event) do
+          :ok -> {:ok, event}
+          {:error, reason} -> {:error, reason}
+        end
+      after
+        RelayClient.close(client)
+      end
+    end
+  end
+
+  defp ensure_event_absent(config, node, event_id) do
+    filter = %{
+      "kinds" => [5001],
+      "#r" => [config.resource],
+      "ids" => [event_id],
+      "limit" => 1
+    }
+
+    case query_events(node, config.client_private_key, filter) do
+      {:ok, []} ->
+        :ok
+
+      {:ok, events} when is_list(events) ->
+        if Enum.any?(events, &(&1["id"] == event_id)) do
+          {:error, {:unexpected_replication, event_id}}
+        else
+          :ok
+        end
+
+      {:error, reason} ->
+        {:error, {:event_absence_query_failed, reason}}
+    end
+  end
+
   defp publish_phase_event(config, node, phase) do
     event =
       %{
@@ -428,7 +579,8 @@ defmodule NodeSyncE2E.Runner do
           [filter],
           [],
           false,
-          nil
+          nil,
+          0
         )
       after
         RelayClient.close(client)
@@ -444,26 +596,32 @@ defmodule NodeSyncE2E.Runner do
          filters,
          events,
          authenticated?,
-         auth_event_id
+         auth_event_id,
+         auth_attempts
        ) do
     receive do
       {:node_sync_e2e_relay_client, ^client, :frame, ["AUTH", challenge]} ->
-        auth_event =
-          auth_event(relay_auth_url, challenge)
-          |> sign_event!(private_key)
+        if auth_attempts >= 5 do
+          {:error, :too_many_auth_challenges}
+        else
+          auth_event =
+            auth_event(relay_auth_url, challenge)
+            |> sign_event!(private_key)
 
-        :ok = RelayClient.send_json(client, ["AUTH", auth_event])
+          :ok = RelayClient.send_json(client, ["AUTH", auth_event])
 
-        authenticated_query(
-          client,
-          relay_auth_url,
-          private_key,
-          subscription_id,
-          filters,
-          events,
-          authenticated?,
-          auth_event["id"]
-        )
+          authenticated_query(
+            client,
+            relay_auth_url,
+            private_key,
+            subscription_id,
+            filters,
+            events,
+            authenticated?,
+            auth_event["id"],
+            auth_attempts + 1
+          )
+        end
 
       {:node_sync_e2e_relay_client, ^client, :frame, ["OK", event_id, true, _message]}
       when event_id == auth_event_id ->
@@ -477,7 +635,8 @@ defmodule NodeSyncE2E.Runner do
           filters,
           events,
           true,
-          nil
+          nil,
+          auth_attempts
         )
 
       {:node_sync_e2e_relay_client, ^client, :frame, ["OK", event_id, false, message]}
@@ -493,7 +652,8 @@ defmodule NodeSyncE2E.Runner do
           filters,
           [event | events],
           authenticated?,
-          auth_event_id
+          auth_event_id,
+          auth_attempts
         )
 
       {:node_sync_e2e_relay_client, ^client, :frame, ["EOSE", ^subscription_id]} ->
@@ -514,7 +674,8 @@ defmodule NodeSyncE2E.Runner do
               filters,
               events,
               authenticated?,
-              auth_event_id
+              auth_event_id,
+              auth_attempts
             )
 
           true ->
@@ -838,9 +999,12 @@ defmodule NodeSyncE2E.Runner do
   defp format_reason({:missing_state_value, key}),
     do: "state file is missing #{key}"
 
+  defp format_reason({:unexpected_replication, event_id}),
+    do: "event #{event_id} should not have replicated but was found"
+
   defp format_reason(:missing_command),
     do:
-      "usage: elixir scripts/node_sync_e2e.exs <bootstrap|publish-resume|verify-resume> --state-file <path>"
+      "usage: elixir scripts/node_sync_e2e.exs <bootstrap|publish-resume|verify-resume|filter-selectivity|sync-stop-restart|bidirectional-sync> --state-file <path>"
 
   defp format_reason(:missing_state_file),
     do: "--state-file is required"

scripts/nostr_seed.mjs

@@ -0,0 +1,185 @@
+#!/usr/bin/env node
+
+// Nostr event seeder — generates and publishes events matching nostr-bench
+// patterns to a relay via WebSocket.
+//
+// Standalone:
+//   node scripts/nostr_seed.mjs --url ws://127.0.0.1:4413/relay --count 10000 [--concurrency 8]
+//
+// As module:
+//   import { seedEvents } from './nostr_seed.mjs';
+//   const result = await seedEvents({ url, count, concurrency: 8 });
+
+import { generateSecretKey, getPublicKey, finalizeEvent } from "nostr-tools/pure";
+import WebSocket from "ws";
+import { parseArgs } from "node:util";
+import { fileURLToPath } from "node:url";
+
+// Matches nostr-bench util.rs:48-66 exactly.
+function generateBenchEvent() {
+  const sk = generateSecretKey();
+  const pk = getPublicKey(sk);
+  const benchTag = `nostr-bench-${Math.floor(Math.random() * 1000)}`;
+
+  return finalizeEvent(
+    {
+      kind: 1,
+      created_at: Math.floor(Date.now() / 1000),
+      content: "This is a message from nostr-bench client",
+      tags: [
+        ["p", pk],
+        [
+          "e",
+          "378f145897eea948952674269945e88612420db35791784abf0616b4fed56ef7",
+        ],
+        ["t", "nostr-bench-"],
+        ["t", benchTag],
+      ],
+    },
+    sk,
+  );
+}
+
+/**
+ * Seed exactly `count` events to a relay.
+ *
+ * Opens `concurrency` parallel WebSocket connections, generates events
+ * matching nostr-bench patterns, sends them, and waits for OK acks.
+ *
+ * @param {object}   opts
+ * @param {string}   opts.url          Relay WebSocket URL
+ * @param {number}   opts.count        Number of events to send
+ * @param {number}  [opts.concurrency] Parallel connections (default 8)
+ * @param {function} [opts.onProgress] Called with (acked_so_far) periodically
+ * @returns {Promise<{sent: number, acked: number, errors: number, elapsed_ms: number}>}
+ */
+export async function seedEvents({ url, count, concurrency = 8, onProgress }) {
+  if (count <= 0) return { sent: 0, acked: 0, errors: 0, elapsed_ms: 0 };
+
+  const startMs = Date.now();
+  let sent = 0;
+  let acked = 0;
+  let errors = 0;
+  let nextToSend = 0;
+
+  // Claim the next event index atomically (single-threaded, but clear intent).
+  function claimNext() {
+    if (nextToSend >= count) return -1;
+    return nextToSend++;
+  }
+
+  async function runWorker() {
+    return new Promise((resolve, reject) => {
+      const ws = new WebSocket(url);
+      let pendingResolve = null;
+      let closed = false;
+
+      ws.on("error", (err) => {
+        if (!closed) {
+          closed = true;
+          reject(err);
+        }
+      });
+
+      ws.on("close", () => {
+        closed = true;
+        if (pendingResolve) pendingResolve();
+        resolve();
+      });
+
+      ws.on("open", () => {
+        sendNext();
+      });
+
+      ws.on("message", (data) => {
+        const msg = data.toString();
+        if (msg.includes('"OK"')) {
+          if (msg.includes("true")) {
+            acked++;
+          } else {
+            errors++;
+          }
+          if (onProgress && (acked + errors) % 10000 === 0) {
+            onProgress(acked);
+          }
+          sendNext();
+        }
+      });
+
+      function sendNext() {
+        if (closed) return;
+        const idx = claimNext();
+        if (idx < 0) {
+          closed = true;
+          ws.close();
+          return;
+        }
+        const event = generateBenchEvent();
+        sent++;
+        ws.send(JSON.stringify(["EVENT", event]));
+      }
+    });
+  }
+
+  const workers = [];
+  const actualConcurrency = Math.min(concurrency, count);
+  for (let i = 0; i < actualConcurrency; i++) {
+    workers.push(runWorker());
+  }
+
+  await Promise.allSettled(workers);
+
+  return {
+    sent,
+    acked,
+    errors,
+    elapsed_ms: Date.now() - startMs,
+  };
+}
+
+// CLI entrypoint
+const isMain =
+  process.argv[1] &&
+  fileURLToPath(import.meta.url).endsWith(process.argv[1].replace(/^.*\//, ""));
+
+if (isMain) {
+  const { values } = parseArgs({
+    options: {
+      url: { type: "string" },
+      count: { type: "string" },
+      concurrency: { type: "string", default: "8" },
+    },
+    strict: false,
+  });
+
+  if (!values.url || !values.count) {
+    console.error(
+      "usage: node scripts/nostr_seed.mjs --url ws://... --count <n> [--concurrency <n>]",
+    );
+    process.exit(1);
+  }
+
+  const count = Number(values.count);
+  const concurrency = Number(values.concurrency);
+
+  if (!Number.isInteger(count) || count < 1) {
+    console.error("--count must be a positive integer");
+    process.exit(1);
+  }
+
+  console.log(
+    `[seed] seeding ${count} events to ${values.url} (concurrency=${concurrency})`,
+  );
+
+  const result = await seedEvents({
+    url: values.url,
+    count,
+    concurrency,
+    onProgress: (n) => {
+      process.stdout.write(`\r[seed] ${n}/${count} acked`);
+    },
+  });
+
+  if (count > 500) process.stdout.write("\n");
+  console.log(JSON.stringify(result));
+}

scripts/run_bench_at_ref.sh

@@ -0,0 +1,125 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+
+usage() {
+  cat <<'EOF'
+usage:
+  ./scripts/run_bench_at_ref.sh <git-ref>
+
+Runs benchmarks for a specific git ref (tag, commit, or branch) and appends
+the results to bench/history.jsonl.
+
+Uses a temporary worktree to avoid disrupting your current working directory.
+
+Arguments:
+  git-ref    Git reference to benchmark (e.g., v0.4.0, abc1234, HEAD~3)
+
+Environment:
+  PARRHESIA_BENCH_RUNS               Number of runs (default: 3)
+  PARRHESIA_BENCH_MACHINE_ID         Machine identifier (default: hostname -s)
+
+All PARRHESIA_BENCH_* knobs from run_bench_compare.sh are forwarded.
+
+Example:
+  # Benchmark a specific tag
+  ./scripts/run_bench_at_ref.sh v0.4.0
+
+  # Benchmark a commit
+  ./scripts/run_bench_at_ref.sh abc1234
+
+  # Quick single-run benchmark
+  PARRHESIA_BENCH_RUNS=1 ./scripts/run_bench_at_ref.sh v0.3.0
+EOF
+}
+
+if [[ $# -eq 0 || "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
+  usage
+  exit 0
+fi
+
+GIT_REF="$1"
+
+# --- Validate ref exists -----------------------------------------------------
+
+if ! git rev-parse --verify "$GIT_REF" >/dev/null 2>&1; then
+  echo "Error: git ref '$GIT_REF' not found" >&2
+  exit 1
+fi
+
+RESOLVED_COMMIT="$(git rev-parse "$GIT_REF")"
+echo "Benchmarking ref: $GIT_REF ($RESOLVED_COMMIT)"
+echo
+
+# --- Setup worktree ----------------------------------------------------------
+
+WORKTREE_DIR="$(mktemp -d)"
+trap 'echo "Cleaning up worktree..."; git worktree remove --force "$WORKTREE_DIR" 2>/dev/null || rm -rf "$WORKTREE_DIR"' EXIT
+
+echo "Creating temporary worktree at $WORKTREE_DIR"
+git worktree add --detach "$WORKTREE_DIR" "$GIT_REF" >/dev/null 2>&1
+
+# --- Run benchmark in worktree -----------------------------------------------
+
+cd "$WORKTREE_DIR"
+
+# Always copy latest benchmark scripts to ensure consistency
+echo "Copying latest benchmark infrastructure from current..."
+mkdir -p scripts bench
+cp "$ROOT_DIR/scripts/run_bench_collect.sh" scripts/
+cp "$ROOT_DIR/scripts/run_bench_compare.sh" scripts/
+cp "$ROOT_DIR/scripts/run_nostr_bench.sh" scripts/
+if [[ -f "$ROOT_DIR/scripts/run_nostr_bench_strfry.sh" ]]; then
+  cp "$ROOT_DIR/scripts/run_nostr_bench_strfry.sh" scripts/
+fi
+if [[ -f "$ROOT_DIR/scripts/run_nostr_bench_nostr_rs_relay.sh" ]]; then
+  cp "$ROOT_DIR/scripts/run_nostr_bench_nostr_rs_relay.sh" scripts/
+fi
+echo
+
+echo "Installing dependencies..."
+mix deps.get
+echo
+
+echo "Running benchmark in worktree..."
+echo
+
+RUNS="${PARRHESIA_BENCH_RUNS:-3}"
+
+# Run the benchmark collect script which will append to history.jsonl
+PARRHESIA_BENCH_RUNS="$RUNS" \
+PARRHESIA_BENCH_MACHINE_ID="${PARRHESIA_BENCH_MACHINE_ID:-}" \
+  ./scripts/run_bench_collect.sh
+
+# --- Copy results back -------------------------------------------------------
+
+echo
+echo "Copying results back to main repository..."
+
+if [[ -f "$WORKTREE_DIR/bench/history.jsonl" ]]; then
+  HISTORY_FILE="$ROOT_DIR/bench/history.jsonl"
+
+  # Get the last line from worktree history (the one just added)
+  NEW_ENTRY="$(tail -n 1 "$WORKTREE_DIR/bench/history.jsonl")"
+
+  # Check if this exact entry already exists in the main history
+  if grep -Fxq "$NEW_ENTRY" "$HISTORY_FILE" 2>/dev/null; then
+    echo "Note: Entry already exists in $HISTORY_FILE, skipping append"
+  else
+    echo "Appending result to $HISTORY_FILE"
+    echo "$NEW_ENTRY" >> "$HISTORY_FILE"
+  fi
+else
+  echo "Warning: No history.jsonl found in worktree" >&2
+  exit 1
+fi
+
+# --- Done --------------------------------------------------------------------
+
+echo
+echo "Benchmark complete for $GIT_REF"
+echo
+echo "To regenerate the chart with updated history:"
+echo "  # The chart generation reads from bench/history.jsonl"
+echo "  # You can manually trigger chart regeneration if needed"

scripts/run_bench_cloud.sh

@@ -0,0 +1,133 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+cd "$ROOT_DIR"
+
+usage() {
+  cat <<'EOF'
+usage:
+  ./scripts/run_bench_cloud.sh [options] [-- extra args for cloud_bench_orchestrate.mjs]
+
+Thin wrapper around scripts/cloud_bench_orchestrate.mjs.
+
+Behavior:
+  - Forwards args directly to the orchestrator.
+  - Adds convenience aliases:
+      --image IMAGE    -> --parrhesia-image IMAGE
+  - Adds smoke defaults when --quick is set (unless already provided):
+      --server-type cx23
+      --client-type cx23
+      --runs 1
+      --clients 1
+      --connect-count 20
+      --connect-rate 20
+      --echo-count 20
+      --echo-rate 20
+      --echo-size 512
+      --event-count 20
+      --event-rate 20
+      --req-count 20
+      --req-rate 20
+      --req-limit 10
+      --keepalive-seconds 2
+
+Flags handled by this wrapper:
+  --quick
+  --image IMAGE
+  -h, --help
+
+Everything else is passed through unchanged.
+
+Examples:
+  just bench cloud
+  just bench cloud --quick
+  just bench cloud --clients 2 --runs 1 --targets parrhesia-memory
+  just bench cloud --image ghcr.io/owner/parrhesia:latest --threads 4
+  just bench cloud --no-monitoring
+  just bench cloud --yes --datacenter auto
+EOF
+}
+
+has_opt() {
+  local key="$1"
+  shift
+  local arg
+  for arg in "$@"; do
+    if [[ "$arg" == "$key" || "$arg" == "$key="* ]]; then
+      return 0
+    fi
+  done
+  return 1
+}
+
+add_default_if_missing() {
+  local key="$1"
+  local value="$2"
+  if ! has_opt "$key" "${ORCH_ARGS[@]}"; then
+    ORCH_ARGS+=("$key" "$value")
+  fi
+}
+
+ORCH_ARGS=()
+QUICK=0
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -h|--help)
+      usage
+      exit 0
+      ;;
+    --quick)
+      QUICK=1
+      ORCH_ARGS+=("--quick")
+      shift
+      ;;
+    --image)
+      if [[ $# -lt 2 ]]; then
+        echo "Missing value for --image" >&2
+        exit 1
+      fi
+      ORCH_ARGS+=("--parrhesia-image" "$2")
+      shift 2
+      ;;
+    --)
+      shift
+      ORCH_ARGS+=("$@")
+      break
+      ;;
+    *)
+      ORCH_ARGS+=("$1")
+      shift
+      ;;
+  esac
+done
+
+if [[ "$QUICK" == "1" ]]; then
+  add_default_if_missing "--server-type" "cx23"
+  add_default_if_missing "--client-type" "cx23"
+  add_default_if_missing "--runs" "1"
+  add_default_if_missing "--clients" "1"
+
+  add_default_if_missing "--connect-count" "20"
+  add_default_if_missing "--connect-rate" "20"
+  add_default_if_missing "--echo-count" "20"
+  add_default_if_missing "--echo-rate" "20"
+  add_default_if_missing "--echo-size" "512"
+  add_default_if_missing "--event-count" "20"
+  add_default_if_missing "--event-rate" "20"
+  add_default_if_missing "--req-count" "20"
+  add_default_if_missing "--req-rate" "20"
+  add_default_if_missing "--req-limit" "10"
+  add_default_if_missing "--keepalive-seconds" "2"
+fi
+
+CMD=(node scripts/cloud_bench_orchestrate.mjs "${ORCH_ARGS[@]}")
+
+printf 'Running cloud bench:\n  %q' "${CMD[0]}"
+for ((i=1; i<${#CMD[@]}; i++)); do
+  printf ' %q' "${CMD[$i]}"
+done
+printf '\n\n'
+
+"${CMD[@]}"

scripts/run_bench_collect.sh

@@ -0,0 +1,115 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+cd "$ROOT_DIR"
+
+usage() {
+  cat <<'EOF'
+usage:
+  ./scripts/run_bench_collect.sh
+
+Runs the benchmark suite and appends results to bench/history.jsonl.
+Does NOT update README.md or regenerate chart.svg.
+
+Use run_bench_update.sh to update the chart and README from collected data.
+
+Environment:
+  PARRHESIA_BENCH_RUNS               Number of runs (default: 3)
+  PARRHESIA_BENCH_MACHINE_ID         Machine identifier (default: hostname -s)
+
+All PARRHESIA_BENCH_* knobs from run_bench_compare.sh are forwarded.
+
+Example:
+  # Collect benchmark data
+  ./scripts/run_bench_collect.sh
+
+  # Later, update chart and README
+  ./scripts/run_bench_update.sh
+EOF
+}
+
+if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
+  usage
+  exit 0
+fi
+
+# --- Configuration -----------------------------------------------------------
+
+BENCH_DIR="$ROOT_DIR/bench"
+HISTORY_FILE="$BENCH_DIR/history.jsonl"
+
+MACHINE_ID="${PARRHESIA_BENCH_MACHINE_ID:-$(hostname -s)}"
+GIT_TAG="$(git describe --tags --abbrev=0 2>/dev/null || echo 'untagged')"
+GIT_COMMIT="$(git rev-parse --short=7 HEAD)"
+TIMESTAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
+RUNS="${PARRHESIA_BENCH_RUNS:-3}"
+
+mkdir -p "$BENCH_DIR"
+
+WORK_DIR="$(mktemp -d)"
+trap 'rm -rf "$WORK_DIR"' EXIT
+
+JSON_OUT="$WORK_DIR/bench_summary.json"
+RAW_OUTPUT="$WORK_DIR/bench_output.txt"
+
+# --- Phase 1: Run benchmarks -------------------------------------------------
+
+echo "Running ${RUNS}-run benchmark suite..."
+
+PARRHESIA_BENCH_RUNS="$RUNS" \
+BENCH_JSON_OUT="$JSON_OUT" \
+  ./scripts/run_bench_compare.sh 2>&1 | tee "$RAW_OUTPUT"
+
+if [[ ! -f "$JSON_OUT" ]]; then
+  echo "Benchmark JSON output not found at $JSON_OUT" >&2
+  exit 1
+fi
+
+# --- Phase 2: Append to history ----------------------------------------------
+
+echo "Appending to history..."
+
+node - "$JSON_OUT" "$TIMESTAMP" "$MACHINE_ID" "$GIT_TAG" "$GIT_COMMIT" "$RUNS" "$HISTORY_FILE" <<'NODE'
+const fs = require("node:fs");
+
+const [, , jsonOut, timestamp, machineId, gitTag, gitCommit, runsStr, historyFile] = process.argv;
+
+const { versions, ...servers } = JSON.parse(fs.readFileSync(jsonOut, "utf8"));
+
+const entry = {
+  schema_version: 2,
+  timestamp,
+  run_id: `local-${timestamp}-${machineId}-${gitCommit}`,
+  machine_id: machineId,
+  git_tag: gitTag,
+  git_commit: gitCommit,
+  runs: Number(runsStr),
+  source: {
+    kind: "local",
+    mode: "run_bench_collect",
+    git_ref: gitTag,
+    git_tag: gitTag,
+    git_commit: gitCommit,
+  },
+  infra: {
+    provider: "local",
+  },
+  versions: versions || {},
+  servers,
+};
+
+fs.appendFileSync(historyFile, JSON.stringify(entry) + "\n", "utf8");
+console.log("  entry: " + gitTag + " (" + gitCommit + ") on " + machineId);
+NODE
+
+# --- Done ---------------------------------------------------------------------
+
+echo
+echo "Benchmark data collected and appended to $HISTORY_FILE"
+echo
+echo "To update chart and README with collected data:"
+echo "  ./scripts/run_bench_update.sh"
+echo
+echo "To update for a specific machine:"
+echo "  ./scripts/run_bench_update.sh <machine_id>"

scripts/run_bench_compare.sh

@@ -298,7 +298,8 @@ for run in $(seq 1 "$RUNS"); do
 
 done
 
-node - "$WORK_DIR" "$RUNS" "$HAS_STRFRY" "$HAS_NOSTR_RS" <<'NODE'
+node - "$WORK_DIR" "$RUNS" "$HAS_STRFRY" "$HAS_NOSTR_RS" \
+     "$PARRHESIA_VERSION" "$STRFRY_VERSION" "$NOSTR_RS_RELAY_VERSION" "$NOSTR_BENCH_VERSION" <<'NODE'
 const fs = require("node:fs");
 const path = require("node:path");
 
@@ -306,6 +307,10 @@ const workDir = process.argv[2];
 const runs = Number(process.argv[3]);
 const hasStrfry = process.argv[4] === "1";
 const hasNostrRs = process.argv[5] === "1";
+const parrhesiaVersion = process.argv[6] || "";
+const strfryVersion = process.argv[7] || "";
+const nostrRsRelayVersion = process.argv[8] || "";
+const nostrBenchVersion = process.argv[9] || "";
 
 function parseLog(filePath) {
   const content = fs.readFileSync(filePath, "utf8");
@@ -502,6 +507,12 @@ if (process.env.BENCH_JSON_OUT) {
     };
   }
 
+  const versions = { parrhesia: parrhesiaVersion };
+  if (hasStrfry && strfryVersion) versions.strfry = strfryVersion;
+  if (hasNostrRs && nostrRsRelayVersion) versions["nostr-rs-relay"] = nostrRsRelayVersion;
+  if (nostrBenchVersion) versions["nostr-bench"] = nostrBenchVersion;
+  jsonSummary.versions = versions;
+
   fs.writeFileSync(
     process.env.BENCH_JSON_OUT,
     JSON.stringify(jsonSummary, null, 2) + "\n",

scripts/run_bench_update.sh

@@ -7,198 +7,356 @@ cd "$ROOT_DIR"
 usage() {
   cat <<'EOF'
 usage:
-  ./scripts/run_bench_update.sh
+  ./scripts/run_bench_update.sh [machine_id|all]
+  ./scripts/run_bench_update.sh --machine <machine_id|all> [--run-id <run_id>]
+  ./scripts/run_bench_update.sh --list
 
-Runs the benchmark suite (3 runs by default), then:
-  1) Appends structured results to bench/history.jsonl
-  2) Generates bench/chart.svg via gnuplot
-  3) Updates the comparison table in README.md
+Regenerates bench/chart.svg and updates the benchmark table in README.md
+from collected data in bench/history.jsonl.
 
-Environment:
-  PARRHESIA_BENCH_RUNS               Number of runs (default: 3)
-  PARRHESIA_BENCH_MACHINE_ID         Machine identifier (default: hostname -s)
-
-All PARRHESIA_BENCH_* knobs from run_bench_compare.sh are forwarded.
+Options:
+  --machine <id|all>    Filter by machine_id (default: hostname -s)
+  --run-id <id>         Filter to an exact run_id
+  --history-file <path> History JSONL file (default: bench/history.jsonl)
+  --list                List available machines and runs, then exit
+  -h, --help
 EOF
 }
 
-if [[ "${1:-}" == "-h" || "${1:-}" == "--help" ]]; then
-  usage
-  exit 0
-fi
-
-# --- Configuration -----------------------------------------------------------
-
 BENCH_DIR="$ROOT_DIR/bench"
 HISTORY_FILE="$BENCH_DIR/history.jsonl"
 CHART_FILE="$BENCH_DIR/chart.svg"
 GNUPLOT_TEMPLATE="$BENCH_DIR/chart.gnuplot"
+README_FILE="$ROOT_DIR/README.md"
 
-MACHINE_ID="${PARRHESIA_BENCH_MACHINE_ID:-$(hostname -s)}"
-GIT_TAG="$(git describe --tags --abbrev=0 2>/dev/null || echo 'untagged')"
-GIT_COMMIT="$(git rev-parse --short=7 HEAD)"
-TIMESTAMP="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
-RUNS="${PARRHESIA_BENCH_RUNS:-3}"
+MACHINE_ID="$(hostname -s)"
+RUN_ID=""
+LIST_ONLY=0
+POSITIONAL_MACHINE=""
 
-mkdir -p "$BENCH_DIR"
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -h|--help)
+      usage
+      exit 0
+      ;;
+    --machine)
+      MACHINE_ID="$2"
+      shift 2
+      ;;
+    --run-id)
+      RUN_ID="$2"
+      shift 2
+      ;;
+    --history-file)
+      HISTORY_FILE="$2"
+      shift 2
+      ;;
+    --list)
+      LIST_ONLY=1
+      shift
+      ;;
+    *)
+      if [[ -z "$POSITIONAL_MACHINE" ]]; then
+        POSITIONAL_MACHINE="$1"
+        shift
+      else
+        echo "Unexpected argument: $1" >&2
+        usage
+        exit 1
+      fi
+      ;;
+  esac
+done
+
+if [[ -n "$POSITIONAL_MACHINE" ]]; then
+  MACHINE_ID="$POSITIONAL_MACHINE"
+fi
+
+if [[ ! -f "$HISTORY_FILE" ]]; then
+  echo "Error: No history file found at $HISTORY_FILE" >&2
+  echo "Run ./scripts/run_bench_collect.sh or ./scripts/run_bench_cloud.sh first" >&2
+  exit 1
+fi
+
+if [[ "$LIST_ONLY" == "1" ]]; then
+  node - "$HISTORY_FILE" <<'NODE'
+const fs = require("node:fs");
+
+const [, , historyFile] = process.argv;
+
+const entries = fs.readFileSync(historyFile, "utf8")
+  .split("\n")
+  .filter((l) => l.trim().length > 0)
+  .map((l) => JSON.parse(l));
+
+if (entries.length === 0) {
+  console.log("No entries in history file.");
+  process.exit(0);
+}
+
+entries.sort((a, b) => b.timestamp.localeCompare(a.timestamp));
+
+const machines = new Map();
+for (const e of entries) {
+  const machineId = e.machine_id || "unknown";
+  const prev = machines.get(machineId);
+  if (!prev) {
+    machines.set(machineId, { count: 1, latest: e });
+  } else {
+    prev.count += 1;
+    if ((e.timestamp || "") > (prev.latest.timestamp || "")) prev.latest = e;
+  }
+}
+
+console.log("Machines:");
+console.log("  machine_id                          entries  latest_timestamp         latest_tag");
+for (const [machineId, info] of [...machines.entries()].sort((a, b) => a[0].localeCompare(b[0]))) {
+  const id = machineId.padEnd(34, " ");
+  const count = String(info.count).padStart(7, " ");
+  const ts = (info.latest.timestamp || "").padEnd(24, " ");
+  const tag = info.latest.git_tag || "";
+  console.log(`  ${id} ${count}  ${ts} ${tag}`);
+}
+
+console.log("\nRuns (newest first):");
+console.log("  timestamp                 run_id                               machine_id                 source  git_tag            targets");
+for (const e of entries) {
+  const ts = (e.timestamp || "").slice(0, 19).padEnd(24, " ");
+  const runId = (e.run_id || "").slice(0, 36).padEnd(36, " ");
+  const machineId = (e.machine_id || "").slice(0, 24).padEnd(24, " ");
+  const source = (e.source?.kind || "").padEnd(6, " ");
+  const tag = (e.git_tag || "").slice(0, 16).padEnd(16, " ");
+  const targets = (e.bench?.targets || Object.keys(e.servers || {})).join(",");
+  console.log(`  ${ts} ${runId} ${machineId} ${source} ${tag} ${targets}`);
+}
+NODE
+  exit 0
+fi
 
 WORK_DIR="$(mktemp -d)"
 trap 'rm -rf "$WORK_DIR"' EXIT
 
-JSON_OUT="$WORK_DIR/bench_summary.json"
-RAW_OUTPUT="$WORK_DIR/bench_output.txt"
+echo "Generating chart (machine=$MACHINE_ID${RUN_ID:+, run_id=$RUN_ID})"
 
-# --- Phase 1: Run benchmarks -------------------------------------------------
-
-echo "Running ${RUNS}-run benchmark suite..."
-
-PARRHESIA_BENCH_RUNS="$RUNS" \
-BENCH_JSON_OUT="$JSON_OUT" \
-  ./scripts/run_bench_compare.sh 2>&1 | tee "$RAW_OUTPUT"
-
-if [[ ! -f "$JSON_OUT" ]]; then
-  echo "Benchmark JSON output not found at $JSON_OUT" >&2
-  exit 1
-fi
-
-# --- Phase 2: Append to history ----------------------------------------------
-
-echo "Appending to history..."
-
-node - "$JSON_OUT" "$TIMESTAMP" "$MACHINE_ID" "$GIT_TAG" "$GIT_COMMIT" "$RUNS" "$HISTORY_FILE" <<'NODE'
-const fs = require("node:fs");
-
-const [, , jsonOut, timestamp, machineId, gitTag, gitCommit, runsStr, historyFile] = process.argv;
-
-const servers = JSON.parse(fs.readFileSync(jsonOut, "utf8"));
-
-const entry = {
-  timestamp,
-  machine_id: machineId,
-  git_tag: gitTag,
-  git_commit: gitCommit,
-  runs: Number(runsStr),
-  servers,
-};
-
-fs.appendFileSync(historyFile, JSON.stringify(entry) + "\n", "utf8");
-console.log("  entry: " + gitTag + " (" + gitCommit + ") on " + machineId);
-NODE
-
-# --- Phase 3: Generate chart --------------------------------------------------
-
-echo "Generating chart..."
-
-node - "$HISTORY_FILE" "$MACHINE_ID" "$WORK_DIR" <<'NODE'
+if ! node - "$HISTORY_FILE" "$MACHINE_ID" "$RUN_ID" "$WORK_DIR" <<'NODE'
 const fs = require("node:fs");
 const path = require("node:path");
 
-const [, , historyFile, machineId, workDir] = process.argv;
+const [, , historyFile, machineId, runId, workDir] = process.argv;
 
-if (!fs.existsSync(historyFile)) {
-  console.log("  no history file, skipping chart generation");
-  process.exit(0);
+function parseSemverTag(tag) {
+  const match = /^v?(\d+)\.(\d+)\.(\d+)$/.exec(tag || "");
+  return match ? match.slice(1).map(Number) : null;
 }
 
-const lines = fs.readFileSync(historyFile, "utf8")
+const all = fs.readFileSync(historyFile, "utf8")
   .split("\n")
-  .filter(l => l.trim().length > 0)
-  .map(l => JSON.parse(l));
+  .filter((l) => l.trim().length > 0)
+  .map((l) => JSON.parse(l));
 
-// Filter to current machine
-const entries = lines.filter(e => e.machine_id === machineId);
-
-if (entries.length === 0) {
-  console.log("  no history entries for machine '" + machineId + "', skipping chart");
-  process.exit(0);
+let selected = all;
+if (runId && runId.length > 0) {
+  selected = all.filter((e) => e.run_id === runId);
+  console.log(`  filtered by run_id: ${runId}`);
+} else if (machineId !== "all") {
+  selected = all.filter((e) => e.machine_id === machineId);
+  console.log(`  filtered to machine: ${machineId}`);
+} else {
+  console.log("  using all machines");
 }
 
-// Sort chronologically, deduplicate by tag (latest wins)
-entries.sort((a, b) => a.timestamp.localeCompare(b.timestamp));
+if (selected.length === 0) {
+  console.error("  no matching history entries");
+  process.exit(1);
+}
+
+selected.sort((a, b) => (a.timestamp || "").localeCompare(b.timestamp || ""));
+
 const byTag = new Map();
-for (const e of entries) {
-  byTag.set(e.git_tag, e);
+for (const e of selected) {
+  byTag.set(e.git_tag || "untagged", e);
 }
 const deduped = [...byTag.values()];
 
-// Determine which non-parrhesia servers are present
-const baselineServerNames = ["strfry", "nostr-rs-relay"];
-const presentBaselines = baselineServerNames.filter(srv =>
-  deduped.some(e => e.servers[srv])
-);
+deduped.sort((a, b) => {
+  const aTag = parseSemverTag(a.git_tag);
+  const bTag = parseSemverTag(b.git_tag);
 
-// Compute averages for baseline servers (constant horizontal lines)
-const baselineAvg = {};
-for (const srv of presentBaselines) {
-  const vals = deduped.filter(e => e.servers[srv]).map(e => e.servers[srv]);
-  baselineAvg[srv] = {};
-  for (const metric of Object.keys(vals[0])) {
-    const valid = vals.map(v => v[metric]).filter(Number.isFinite);
-    baselineAvg[srv][metric] = valid.length > 0
-      ? valid.reduce((a, b) => a + b, 0) / valid.length
-      : NaN;
+  if (aTag && bTag) {
+    return aTag[0] - bTag[0] || aTag[1] - bTag[1] || aTag[2] - bTag[2];
+  }
+
+  return (a.git_tag || "").localeCompare(b.git_tag || "", undefined, { numeric: true });
+});
+
+const primaryServerNames = new Set(["parrhesia-pg", "parrhesia-memory"]);
+const preferredBaselineOrder = ["strfry", "nostr-rs-relay", "nostream", "haven"];
+
+const discoveredBaselines = new Set();
+for (const e of deduped) {
+  for (const serverName of Object.keys(e.servers || {})) {
+    if (!primaryServerNames.has(serverName)) {
+      discoveredBaselines.add(serverName);
+    }
   }
 }
 
-// Metrics to chart
-const chartMetrics = [
-  { key: "event_tps",      label: "Event Throughput (TPS) — higher is better",    file: "event_tps.tsv",      ylabel: "TPS" },
-  { key: "req_tps",        label: "Req Throughput (TPS) — higher is better",      file: "req_tps.tsv",        ylabel: "TPS" },
-  { key: "echo_tps",       label: "Echo Throughput (TPS) — higher is better",     file: "echo_tps.tsv",       ylabel: "TPS" },
-  { key: "connect_avg_ms", label: "Connect Avg Latency (ms) — lower is better",   file: "connect_avg_ms.tsv", ylabel: "ms"  },
+const presentBaselines = [
+  ...preferredBaselineOrder.filter((srv) => discoveredBaselines.has(srv)),
+  ...[...discoveredBaselines].filter((srv) => !preferredBaselineOrder.includes(srv)).sort((a, b) => a.localeCompare(b)),
 ];
 
-// Write per-metric TSV files
-for (const cm of chartMetrics) {
-  const header = ["tag", "parrhesia-pg", "parrhesia-memory"];
-  for (const srv of presentBaselines) header.push(srv);
+// --- Colour palette per server: [empty, warm, hot] ---
+const serverColours = {
+  "parrhesia-pg":     ["#93c5fd", "#3b82f6", "#1e40af"],
+  "parrhesia-memory": ["#86efac", "#22c55e", "#166534"],
+  "strfry":           ["#fdba74", "#f97316", "#9a3412"],
+  "nostr-rs-relay":   ["#fca5a5", "#ef4444", "#991b1b"],
+  "nostream":         ["#d8b4fe", "#a855f7", "#6b21a8"],
+  "haven":            ["#fde68a", "#eab308", "#854d0e"],
+};
 
-  const rows = [header.join("\t")];
-  for (const e of deduped) {
-    const row = [
-      e.git_tag,
-      e.servers["parrhesia-pg"]?.[cm.key] ?? "NaN",
-      e.servers["parrhesia-memory"]?.[cm.key] ?? "NaN",
-    ];
-    for (const srv of presentBaselines) {
-      row.push(baselineAvg[srv]?.[cm.key] ?? "NaN");
-    }
-    rows.push(row.join("\t"));
+const levelStyles = [
+  /* empty */ { dt: 3, pt: 6, ps: 0.7, lw: 1.5 },
+  /* warm  */ { dt: 2, pt: 8, ps: 0.8, lw: 1.5 },
+  /* hot   */ { dt: 1, pt: 7, ps: 1.0, lw: 2 },
+];
+
+const levels = ["empty", "warm", "hot"];
+
+const shortLabel = {
+  "parrhesia-pg": "pg", "parrhesia-memory": "mem",
+  "strfry": "strfry", "nostr-rs-relay": "nostr-rs",
+  "nostream": "nostream", "haven": "haven",
+};
+
+const allServers = ["parrhesia-pg", "parrhesia-memory", ...presentBaselines];
+
+function isPhased(e) {
+  for (const srv of Object.values(e.servers || {})) {
+    if (srv.event_empty_tps !== undefined) return true;
   }
-
-  fs.writeFileSync(path.join(workDir, cm.file), rows.join("\n") + "\n", "utf8");
+  return false;
 }
 
-// Generate gnuplot plot commands (handles variable column counts)
-const serverLabels = ["parrhesia-pg", "parrhesia-memory"];
-for (const srv of presentBaselines) serverLabels.push(srv + " (avg)");
+// Build phased key: "event_tps" + "empty" → "event_empty_tps"
+function phasedKey(base, level) {
+  const idx = base.lastIndexOf("_");
+  return `${base.slice(0, idx)}_${level}_${base.slice(idx + 1)}`;
+}
 
+// --- Emit linetype definitions (server × level) ---
 const plotLines = [];
-for (const cm of chartMetrics) {
-  const dataFile = `data_dir."/${cm.file}"`;
-  plotLines.push(`set title "${cm.label}"`);
-  plotLines.push(`set ylabel "${cm.ylabel}"`);
-
-  const plotParts = [];
-  // Column 2 = parrhesia-pg, 3 = parrhesia-memory, 4+ = baselines
-  plotParts.push(`${dataFile} using 0:2:xtic(1) lt 1 title "${serverLabels[0]}"`);
-  plotParts.push(`'' using 0:3 lt 2 title "${serverLabels[1]}"`);
-  for (let i = 0; i < presentBaselines.length; i++) {
-    plotParts.push(`'' using 0:${4 + i} lt ${3 + i} title "${serverLabels[2 + i]}"`);
+for (let si = 0; si < allServers.length; si++) {
+  const colours = serverColours[allServers[si]] || ["#888888", "#555555", "#222222"];
+  for (let li = 0; li < 3; li++) {
+    const s = levelStyles[li];
+    plotLines.push(
+      `set linetype ${si * 3 + li + 1} lc rgb "${colours[li]}" lw ${s.lw} pt ${s.pt} ps ${s.ps} dt ${s.dt}`
+    );
   }
+}
+plotLines.push("");
 
-  plotLines.push("plot " + plotParts.join(", \\\n     "));
-  plotLines.push("");
+// Panel definitions — order matches 4x2 grid (left-to-right, top-to-bottom)
+const panels = [
+  { kind: "simple", key: "echo_tps",       label: "Echo Throughput (TPS) — higher is better",    file: "echo_tps.tsv",       ylabel: "TPS" },
+  { kind: "simple", key: "echo_mibs",      label: "Echo Throughput (MiB/s) — higher is better",  file: "echo_mibs.tsv",      ylabel: "MiB/s" },
+  { kind: "fill",   base: "event_tps",     label: "Event Throughput (TPS) — higher is better",   file: "event_tps.tsv",      ylabel: "TPS" },
+  { kind: "fill",   base: "event_mibs",    label: "Event Throughput (MiB/s) — higher is better", file: "event_mibs.tsv",     ylabel: "MiB/s" },
+  { kind: "fill",   base: "req_tps",       label: "Req Throughput (TPS) — higher is better",     file: "req_tps.tsv",        ylabel: "TPS" },
+  { kind: "fill",   base: "req_mibs",      label: "Req Throughput (MiB/s) — higher is better",   file: "req_mibs.tsv",       ylabel: "MiB/s" },
+  { kind: "simple", key: "connect_avg_ms", label: "Connect Avg Latency (ms) — lower is better",  file: "connect_avg_ms.tsv", ylabel: "ms" },
+];
+
+for (const panel of panels) {
+  if (panel.kind === "simple") {
+    // One column per server
+    const header = ["tag", ...allServers.map((s) => shortLabel[s] || s)];
+    const rows = [header.join("\t")];
+    for (const e of deduped) {
+      const row = [e.git_tag || "untagged"];
+      for (const srv of allServers) {
+        row.push(e.servers?.[srv]?.[panel.key] ?? "NaN");
+      }
+      rows.push(row.join("\t"));
+    }
+    fs.writeFileSync(path.join(workDir, panel.file), rows.join("\n") + "\n", "utf8");
+
+    // Plot: one series per server, using its "hot" linetype
+    const dataFile = `data_dir."/${panel.file}"`;
+    plotLines.push(`set title "${panel.label}"`);
+    plotLines.push(`set ylabel "${panel.ylabel}"`);
+    const parts = allServers.map((srv, si) => {
+      const src = si === 0 ? dataFile : "''";
+      const xtic = si === 0 ? ":xtic(1)" : "";
+      return `${src} using 0:${si + 2}${xtic} lt ${si * 3 + 3} title "${shortLabel[srv] || srv}"`;
+    });
+    plotLines.push("plot " + parts.join(", \\\n     "));
+    plotLines.push("");
+
+  } else {
+    // Three columns per server (empty, warm, hot)
+    const header = ["tag"];
+    for (const srv of allServers) {
+      const sl = shortLabel[srv] || srv;
+      for (const lvl of levels) header.push(`${sl}-${lvl}`);
+    }
+    const rows = [header.join("\t")];
+    for (const e of deduped) {
+      const row = [e.git_tag || "untagged"];
+      const phased = isPhased(e);
+      for (const srv of allServers) {
+        const d = e.servers?.[srv];
+        if (!d) { row.push("NaN", "NaN", "NaN"); continue; }
+        if (phased) {
+          for (const lvl of levels) row.push(d[phasedKey(panel.base, lvl)] ?? "NaN");
+        } else {
+          row.push("NaN", d[panel.base] ?? "NaN", "NaN"); // flat → warm only
+        }
+      }
+      rows.push(row.join("\t"));
+    }
+    fs.writeFileSync(path.join(workDir, panel.file), rows.join("\n") + "\n", "utf8");
+
+    // Plot: three series per server (empty/warm/hot)
+    const dataFile = `data_dir."/${panel.file}"`;
+    plotLines.push(`set title "${panel.label}"`);
+    plotLines.push(`set ylabel "${panel.ylabel}"`);
+    const parts = [];
+    let first = true;
+    for (let si = 0; si < allServers.length; si++) {
+      const label = shortLabel[allServers[si]] || allServers[si];
+      for (let li = 0; li < 3; li++) {
+        const src = first ? dataFile : "''";
+        const xtic = first ? ":xtic(1)" : "";
+        const col = 2 + si * 3 + li;
+        parts.push(`${src} using 0:${col}${xtic} lt ${si * 3 + li + 1} title "${label} (${levels[li]})"`);
+        first = false;
+      }
+    }
+    plotLines.push("plot " + parts.join(", \\\n     "));
+    plotLines.push("");
+  }
 }
 
-fs.writeFileSync(
-  path.join(workDir, "plot_commands.gnuplot"),
-  plotLines.join("\n") + "\n",
-  "utf8"
-);
+fs.writeFileSync(path.join(workDir, "plot_commands.gnuplot"), plotLines.join("\n") + "\n", "utf8");
 
-console.log("  " + deduped.length + " tag(s), " + presentBaselines.length + " baseline server(s)");
+const latestForReadme = [...selected]
+  .sort((a, b) => (b.timestamp || "").localeCompare(a.timestamp || ""))
+  .find((e) => e.servers?.["parrhesia-pg"] && e.servers?.["parrhesia-memory"]);
+
+if (latestForReadme) {
+  fs.writeFileSync(path.join(workDir, "latest_entry.json"), JSON.stringify(latestForReadme), "utf8");
+}
+
+console.log(`  selected=${selected.length}, series_tags=${deduped.length}, baselines=${presentBaselines.length}`);
 NODE
+then
+  echo "No matching data for chart/update" >&2
+  exit 1
+fi
 
 if [[ -f "$WORK_DIR/plot_commands.gnuplot" ]]; then
   gnuplot \
@@ -207,25 +365,46 @@ if [[ -f "$WORK_DIR/plot_commands.gnuplot" ]]; then
     "$GNUPLOT_TEMPLATE"
   echo "  chart written to $CHART_FILE"
 else
-  echo "  chart generation skipped (no data for this machine)"
+  echo "  chart generation skipped"
 fi
 
-# --- Phase 4: Update README.md -----------------------------------------------
+echo "Updating README.md with latest benchmark..."
 
-echo "Updating README.md..."
+if [[ ! -f "$WORK_DIR/latest_entry.json" ]]; then
+  echo "Warning: no selected entry contains both parrhesia-pg and parrhesia-memory; skipping README table update" >&2
+  echo
+  echo "Benchmark rendering complete. Files updated:"
+  echo "  $CHART_FILE"
+  echo
+  exit 0
+fi
 
-node - "$JSON_OUT" "$ROOT_DIR/README.md" <<'NODE'
+node - "$WORK_DIR/latest_entry.json" "$README_FILE" <<'NODE'
 const fs = require("node:fs");
 
-const [, , jsonOut, readmePath] = process.argv;
-
-const servers = JSON.parse(fs.readFileSync(jsonOut, "utf8"));
-const readme = fs.readFileSync(readmePath, "utf8");
+const [, , entryPath, readmePath] = process.argv;
+const entry = JSON.parse(fs.readFileSync(entryPath, "utf8"));
+const servers = entry.servers || {};
 
 const pg = servers["parrhesia-pg"];
 const mem = servers["parrhesia-memory"];
-const strfry = servers["strfry"];
-const nostrRs = servers["nostr-rs-relay"];
+
+if (!pg || !mem) {
+  console.error("Selected entry is missing parrhesia-pg or parrhesia-memory");
+  process.exit(1);
+}
+
+// Detect phased entries — use hot fill level as headline metric
+const phased = pg.event_empty_tps !== undefined;
+
+// For phased entries, resolve "event_tps" → "event_hot_tps" etc.
+function resolveKey(key) {
+  if (!phased) return key;
+  const fillKeys = ["event_tps", "event_mibs", "req_tps", "req_mibs"];
+  if (!fillKeys.includes(key)) return key;
+  const idx = key.lastIndexOf("_");
+  return `${key.slice(0, idx)}_hot_${key.slice(idx + 1)}`;
+}
 
 function toFixed(v, d = 2) {
   return Number.isFinite(v) ? v.toFixed(d) : "n/a";
@@ -238,44 +417,56 @@ function ratio(base, other) {
 
 function boldIf(ratioStr, lowerIsBetter) {
   if (ratioStr === "n/a") return ratioStr;
-  const num = parseFloat(ratioStr);
+  const num = Number.parseFloat(ratioStr);
+  if (!Number.isFinite(num)) return ratioStr;
   const better = lowerIsBetter ? num < 1 : num > 1;
-  return better ? "**" + ratioStr + "**" : ratioStr;
+  return better ? `**${ratioStr}**` : ratioStr;
 }
 
+const fillNote = phased ? " (hot fill level)" : "";
+
 const metricRows = [
-  ["connect avg latency (ms) \u2193", "connect_avg_ms", true],
-  ["connect max latency (ms) \u2193", "connect_max_ms", true],
-  ["echo throughput (TPS) \u2191",    "echo_tps",       false],
-  ["echo throughput (MiB/s) \u2191",  "echo_mibs",      false],
-  ["event throughput (TPS) \u2191",   "event_tps",      false],
-  ["event throughput (MiB/s) \u2191", "event_mibs",     false],
-  ["req throughput (TPS) \u2191",     "req_tps",        false],
-  ["req throughput (MiB/s) \u2191",   "req_mibs",       false],
+  ["connect avg latency (ms) ↓", "connect_avg_ms", true],
+  ["connect max latency (ms) ↓", "connect_max_ms", true],
+  ["echo throughput (TPS) ↑", "echo_tps", false],
+  ["echo throughput (MiB/s) ↑", "echo_mibs", false],
+  [`event throughput (TPS)${fillNote} ↑`, "event_tps", false],
+  [`event throughput (MiB/s)${fillNote} ↑`, "event_mibs", false],
+  [`req throughput (TPS)${fillNote} ↑`, "req_tps", false],
+  [`req throughput (MiB/s)${fillNote} ↑`, "req_mibs", false],
 ];
 
-const hasStrfry = !!strfry;
-const hasNostrRs = !!nostrRs;
+const preferredComparisonOrder = ["strfry", "nostr-rs-relay", "nostream", "haven"];
+const discoveredComparisons = Object.keys(servers).filter(
+  (name) => name !== "parrhesia-pg" && name !== "parrhesia-memory",
+);
 
-// Build header
-const header = ["metric", "parrhesia-pg", "parrhesia-mem"];
-if (hasStrfry) header.push("strfry");
-if (hasNostrRs) header.push("nostr-rs-relay");
-header.push("mem/pg");
-if (hasStrfry) header.push("strfry/pg");
-if (hasNostrRs) header.push("nostr-rs/pg");
+const comparisonServers = [
+  ...preferredComparisonOrder.filter((name) => discoveredComparisons.includes(name)),
+  ...discoveredComparisons.filter((name) => !preferredComparisonOrder.includes(name)).sort((a, b) => a.localeCompare(b)),
+];
+
+const header = ["metric", "parrhesia-pg", "parrhesia-mem", ...comparisonServers, "mem/pg"];
+for (const serverName of comparisonServers) {
+  header.push(`${serverName}/pg`);
+}
 
 const alignRow = ["---"];
-for (let i = 1; i < header.length; i++) alignRow.push("---:");
+for (let i = 1; i < header.length; i += 1) alignRow.push("---:");
 
 const rows = metricRows.map(([label, key, lowerIsBetter]) => {
-  const row = [label, toFixed(pg[key]), toFixed(mem[key])];
-  if (hasStrfry) row.push(toFixed(strfry[key]));
-  if (hasNostrRs) row.push(toFixed(nostrRs[key]));
+  const rk = resolveKey(key);
+  const row = [label, toFixed(pg[rk]), toFixed(mem[rk])];
 
-  row.push(boldIf(ratio(pg[key], mem[key]), lowerIsBetter));
-  if (hasStrfry) row.push(boldIf(ratio(pg[key], strfry[key]), lowerIsBetter));
-  if (hasNostrRs) row.push(boldIf(ratio(pg[key], nostrRs[key]), lowerIsBetter));
+  for (const serverName of comparisonServers) {
+    row.push(toFixed(servers?.[serverName]?.[rk]));
+  }
+
+  row.push(boldIf(ratio(pg[rk], mem[rk]), lowerIsBetter));
+
+  for (const serverName of comparisonServers) {
+    row.push(boldIf(ratio(pg[rk], servers?.[serverName]?.[rk]), lowerIsBetter));
+  }
 
   return row;
 });
@@ -283,12 +474,12 @@ const rows = metricRows.map(([label, key, lowerIsBetter]) => {
 const tableLines = [
   "| " + header.join(" | ") + " |",
   "| " + alignRow.join(" | ") + " |",
-  ...rows.map(r => "| " + r.join(" | ") + " |"),
+  ...rows.map((r) => "| " + r.join(" | ") + " |"),
 ];
 
-// Replace the first markdown table in the ## Benchmark section
-const readmeLines = readme.split("\n");
-const benchIdx = readmeLines.findIndex(l => /^## Benchmark/.test(l));
+const readme = fs.readFileSync(readmePath, "utf8");
+const lines = readme.split("\n");
+const benchIdx = lines.findIndex((l) => /^## Benchmark/.test(l));
 if (benchIdx === -1) {
   console.error("Could not find '## Benchmark' section in README.md");
   process.exit(1);
@@ -296,8 +487,8 @@ if (benchIdx === -1) {
 
 let tableStart = -1;
 let tableEnd = -1;
-for (let i = benchIdx + 1; i < readmeLines.length; i++) {
-  if (readmeLines[i].startsWith("|")) {
+for (let i = benchIdx + 1; i < lines.length; i += 1) {
+  if (lines[i].startsWith("|")) {
     if (tableStart === -1) tableStart = i;
     tableEnd = i;
   } else if (tableStart !== -1) {
@@ -310,20 +501,19 @@ if (tableStart === -1) {
   process.exit(1);
 }
 
-const before = readmeLines.slice(0, tableStart);
-const after = readmeLines.slice(tableEnd + 1);
-const updated = [...before, ...tableLines, ...after].join("\n");
+const updated = [
+  ...lines.slice(0, tableStart),
+  ...tableLines,
+  ...lines.slice(tableEnd + 1),
+].join("\n");
 
 fs.writeFileSync(readmePath, updated, "utf8");
-console.log("  table updated (" + tableLines.length + " rows)");
+console.log(`  table updated (${tableLines.length} rows)`);
 NODE
 
-# --- Done ---------------------------------------------------------------------
-
 echo
-echo "Benchmark update complete. Files changed:"
-echo "  $HISTORY_FILE"
+echo "Benchmark rendering complete. Files updated:"
 echo "  $CHART_FILE"
-echo "  $ROOT_DIR/README.md"
+echo "  $README_FILE"
 echo
 echo "Review with: git diff"

scripts/run_node_sync_docker_e2e.sh

@@ -108,4 +108,8 @@ docker compose -f "$COMPOSE_FILE" up -d parrhesia-b
 wait_for_health "$NODE_B_HTTP_URL" "Node B"
 run_runner verify-resume
 
+run_runner filter-selectivity
+run_runner sync-stop-restart
+run_runner bidirectional-sync
+
 printf 'node-sync-e2e docker run completed\nstate: %s\n' "$STATE_FILE"

scripts/run_node_sync_e2e.sh

@@ -115,7 +115,7 @@ wait_for_health() {
       return
     fi
 
-    sleep 0.1
+    sleep 0.2
   done
 
   echo "${label} did not become healthy on port ${port}" >&2
@@ -224,4 +224,8 @@ start_node \
 wait_for_health "$NODE_B_PORT" "Node B"
 run_runner verify-resume
 
+run_runner filter-selectivity
+run_runner sync-stop-restart
+run_runner bidirectional-sync
+
 printf 'node-sync-e2e local run completed\nlogs: %s\n' "$LOG_DIR"

test/marmot_e2e/marmot_client_e2e.test.mjs

@@ -11,6 +11,7 @@ import {
   MarmotClient,
   KeyPackageStore,
   KeyValueGroupStateBackend,
+  Proposals,
   createKeyPackageRelayListEvent,
   deserializeApplicationData,
   extractMarmotGroupData,
@@ -142,6 +143,129 @@ function createClient(account, network) {
   });
 }
 
+/**
+ * Bootstraps a group with one admin and `count` invited members.
+ */
+async function createGroupWithMembers(network, count) {
+  const adminAccount = PrivateKeyAccount.generateNew();
+  const adminPubkey = await adminAccount.signer.getPublicKey();
+  const adminClient = createClient(adminAccount, network);
+
+  const adminRelayList = await adminAccount.signer.signEvent(
+    createKeyPackageRelayListEvent({
+      pubkey: adminPubkey,
+      relays: [relayWsUrl],
+      client: "parrhesia-marmot-e2e",
+    }),
+  );
+  await network.publish([relayWsUrl], adminRelayList);
+
+  const memberInfos = [];
+  for (let i = 0; i < count; i++) {
+    const account = PrivateKeyAccount.generateNew();
+    const pubkey = await account.signer.getPublicKey();
+    const client = createClient(account, network);
+
+    const relayList = await account.signer.signEvent(
+      createKeyPackageRelayListEvent({
+        pubkey,
+        relays: [relayWsUrl],
+        client: "parrhesia-marmot-e2e",
+      }),
+    );
+    await network.publish([relayWsUrl], relayList);
+    await client.keyPackages.create({
+      relays: [relayWsUrl],
+      client: "parrhesia-marmot-e2e",
+    });
+
+    memberInfos.push({ account, pubkey, client });
+  }
+
+  const adminGroup = await adminClient.createGroup(
+    `E2E Group ${randomUUID()}`,
+    { relays: [relayWsUrl], adminPubkeys: [adminPubkey] },
+  );
+
+  const members = [];
+  for (const mi of memberInfos) {
+    const keyPackageEvents = await network.request([relayWsUrl], [
+      { kinds: [KEY_PACKAGE_KIND], authors: [mi.pubkey], limit: 1 },
+    ]);
+    assert.equal(keyPackageEvents.length, 1, `key package missing for ${mi.pubkey}`);
+    await adminGroup.inviteByKeyPackageEvent(keyPackageEvents[0]);
+
+    const giftWraps = await requestGiftWrapsWithAuth({
+      relayUrl: relayWsUrl,
+      relayHttpUrl,
+      signer: mi.account.signer,
+      recipientPubkey: mi.pubkey,
+    });
+    assert.ok(giftWraps.length >= 1, `gift wrap missing for ${mi.pubkey}`);
+
+    const inviteReader = new InviteReader({
+      signer: mi.account.signer,
+      store: {
+        received: new MemoryBackend(),
+        unread: new MemoryBackend(),
+        seen: new MemoryBackend(),
+      },
+    });
+    await inviteReader.ingestEvents(giftWraps);
+    const invites = await inviteReader.decryptGiftWraps();
+    assert.equal(invites.length, 1);
+
+    const { group } = await mi.client.joinGroupFromWelcome({
+      welcomeRumor: invites[0],
+    });
+
+    members.push({ account: mi.account, pubkey: mi.pubkey, client: mi.client, group });
+  }
+
+  return {
+    admin: { account: adminAccount, pubkey: adminPubkey, client: adminClient, group: adminGroup },
+    members,
+  };
+}
+
+function getNostrGroupId(group) {
+  const data = extractMarmotGroupData(group.state);
+  assert.ok(data, "MarmotGroupData should exist on group");
+  return bytesToHex(data.nostrGroupId);
+}
+
+async function fetchGroupEvents(network, group) {
+  const nostrGroupId = getNostrGroupId(group);
+  return network.request([relayWsUrl], [
+    { kinds: [GROUP_EVENT_KIND], "#h": [nostrGroupId], limit: 100 },
+  ]);
+}
+
+async function countEvents(relayUrl, filters) {
+  const relay = await openRelay(relayUrl, 5_000);
+  const subId = randomSubId("count");
+  const filtersArray = Array.isArray(filters) ? filters : [filters];
+
+  try {
+    relay.send(["COUNT", subId, ...filtersArray]);
+
+    while (true) {
+      const frame = await relay.nextFrame(5_000);
+      if (!Array.isArray(frame)) continue;
+
+      if (frame[0] === "COUNT" && frame[1] === subId) {
+        return frame[2];
+      }
+
+      if (frame[0] === "CLOSED" && frame[1] === subId) {
+        throw new Error(`COUNT closed: ${frame[2]}`);
+      }
+    }
+  } finally {
+    await relay.close();
+  }
+}
+
 function computeEventId(event) {
   const payload = [
     0,
@@ -603,3 +727,230 @@ test("admin invites user, user joins, and message round-trip decrypts", async ()
     "admin should decrypt invitee application message",
   );
 });
+
+test("sendChatMessage convenience API works", async () => {
+  const network = new RelayNetwork(relayWsUrl, relayHttpUrl);
+  const { admin, members } = await createGroupWithMembers(network, 1);
+  const invitee = members[0];
+
+  const content = `chat-msg-${randomUUID()}`;
+  await invitee.group.sendChatMessage(content);
+
+  const groupEvents = await fetchGroupEvents(network, admin.group);
+
+  const decrypted = [];
+  for await (const result of admin.group.ingest(groupEvents)) {
+    if (result.kind === "processed" && result.result.kind === "applicationMessage") {
+      decrypted.push(deserializeApplicationData(result.result.message));
+    }
+  }
+
+  assert.ok(
+    decrypted.some((r) => r.content === content && r.kind === 9),
+    "admin should decrypt kind 9 chat message from invitee",
+  );
+});
+
+test("admin removes member from group", async () => {
+  const network = new RelayNetwork(relayWsUrl, relayHttpUrl);
+  const { admin, members } = await createGroupWithMembers(network, 2);
+  const [user1, user2] = members;
+
+  // proposeRemoveUser returns ProposalRemove[] — use propose() which handles arrays
+  await admin.group.propose(Proposals.proposeRemoveUser(user2.pubkey));
+  await admin.group.commit();
+
+  // user1 ingests the removal commit
+  const groupEvents = await fetchGroupEvents(network, admin.group);
+
+  let user1Processed = false;
+  for await (const result of user1.group.ingest(groupEvents)) {
+    if (result.kind === "processed" && result.result.kind === "newState") {
+      user1Processed = true;
+    }
+  }
+  assert.ok(user1Processed, "user1 should process the removal commit");
+
+  // user1 can still send a message that admin decrypts
+  const msg = `post-removal-${randomUUID()}`;
+  await user1.group.sendChatMessage(msg);
+
+  const updatedEvents = await fetchGroupEvents(network, admin.group);
+
+  const decrypted = [];
+  for await (const result of admin.group.ingest(updatedEvents)) {
+    if (result.kind === "processed" && result.result.kind === "applicationMessage") {
+      decrypted.push(deserializeApplicationData(result.result.message));
+    }
+  }
+
+  assert.ok(
+    decrypted.some((r) => r.content === msg),
+    "admin should decrypt message from user1 after user2 removal",
+  );
+});
+
+test("group metadata update via proposal", async () => {
+  const network = new RelayNetwork(relayWsUrl, relayHttpUrl);
+  const { admin, members } = await createGroupWithMembers(network, 1);
+  const invitee = members[0];
+
+  const updatedName = `Updated-${randomUUID()}`;
+  await admin.group.commit({
+    extraProposals: [Proposals.proposeUpdateMetadata({ name: updatedName })],
+  });
+
+  // Admin sees the updated name locally
+  const adminGroupData = extractMarmotGroupData(admin.group.state);
+  assert.equal(adminGroupData.name, updatedName);
+
+  // Invitee ingests the commit and sees updated metadata
+  const groupEvents = await fetchGroupEvents(network, admin.group);
+
+  for await (const _result of invitee.group.ingest(groupEvents)) {
+    // drain ingest
+  }
+
+  const inviteeGroupData = extractMarmotGroupData(invitee.group.state);
+  assert.equal(inviteeGroupData.name, updatedName, "invitee should see updated group name");
+});
+
+test("member self-update rotates leaf keys (forward secrecy)", async () => {
+  const network = new RelayNetwork(relayWsUrl, relayHttpUrl);
+  const { admin, members } = await createGroupWithMembers(network, 1);
+  const invitee = members[0];
+
+  const epochBefore = admin.group.state.groupContext.epoch;
+
+  await invitee.group.selfUpdate();
+
+  // Admin ingests the self-update commit
+  const groupEvents = await fetchGroupEvents(network, admin.group);
+
+  for await (const result of admin.group.ingest(groupEvents)) {
+    // drain
+  }
+
+  const epochAfter = admin.group.state.groupContext.epoch;
+  assert.ok(epochAfter > epochBefore, "admin epoch should advance after self-update commit");
+
+  // Both parties can still exchange messages after key rotation
+  const msg = `after-selfupdate-${randomUUID()}`;
+  await invitee.group.sendChatMessage(msg);
+
+  const updatedEvents = await fetchGroupEvents(network, admin.group);
+
+  const decrypted = [];
+  for await (const result of admin.group.ingest(updatedEvents)) {
+    if (result.kind === "processed" && result.result.kind === "applicationMessage") {
+      decrypted.push(deserializeApplicationData(result.result.message));
+    }
+  }
+
+  assert.ok(
+    decrypted.some((r) => r.content === msg),
+    "admin should decrypt message sent after self-update",
+  );
+});
+
+test("member leaves group voluntarily", async () => {
+  const network = new RelayNetwork(relayWsUrl, relayHttpUrl);
+  const { admin, members } = await createGroupWithMembers(network, 2);
+  const [stayer, leaver] = members;
+
+  // Leaver publishes self-remove proposals and destroys local state
+  await leaver.group.leave();
+
+  // Admin ingests the leave proposals (they become unapplied)
+  const groupEventsForAdmin = await fetchGroupEvents(network, admin.group);
+  for await (const _result of admin.group.ingest(groupEventsForAdmin)) {
+    // drain
+  }
+
+  // Admin commits all unapplied proposals (the leave removals)
+  await admin.group.commit();
+
+  // Stayer ingests everything and the group continues
+  const allEvents = await fetchGroupEvents(network, admin.group);
+  for await (const _result of stayer.group.ingest(allEvents)) {
+    // drain
+  }
+
+  const msgAfterLeave = `after-leave-${randomUUID()}`;
+  await stayer.group.sendChatMessage(msgAfterLeave);
+
+  const latestEvents = await fetchGroupEvents(network, admin.group);
+
+  const decrypted = [];
+  for await (const result of admin.group.ingest(latestEvents)) {
+    if (result.kind === "processed" && result.result.kind === "applicationMessage") {
+      decrypted.push(deserializeApplicationData(result.result.message));
+    }
+  }
+
+  assert.ok(
+    decrypted.some((r) => r.content === msgAfterLeave),
+    "group should continue functioning after member departure",
+  );
+});
+
+test("NIP-45 COUNT returns accurate event count", async () => {
+  const account = PrivateKeyAccount.generateNew();
+  const pubkey = await account.signer.getPublicKey();
+  const tag = `count-test-${randomUUID()}`;
+
+  for (let i = 0; i < 3; i++) {
+    const event = {
+      kind: 1,
+      pubkey,
+      created_at: unixNow() + i,
+      tags: [["t", tag]],
+      content: `count-event-${i}`,
+    };
+    event.id = computeEventId(event);
+    const signed = await account.signer.signEvent(event);
+    const frame = await publishEvent(relayWsUrl, signed);
+    assert.equal(frame[2], true, `publish of event ${i} failed: ${frame[3]}`);
+  }
+
+  const result = await countEvents(relayWsUrl, { kinds: [1], "#t": [tag] });
+  assert.equal(result.count, 3, "COUNT should report 3 events");
+});
+
+test("NIP-9 event deletion removes event from relay", async () => {
+  const account = PrivateKeyAccount.generateNew();
+  const pubkey = await account.signer.getPublicKey();
+
+  const original = {
+    kind: 1,
+    pubkey,
+    created_at: unixNow(),
+    tags: [],
+    content: `to-be-deleted-${randomUUID()}`,
+  };
+  original.id = computeEventId(original);
+  const signedOriginal = await account.signer.signEvent(original);
+  const publishFrame = await publishEvent(relayWsUrl, signedOriginal);
+  assert.equal(publishFrame[2], true, `original publish failed: ${publishFrame[3]}`);
+
+  // Verify it exists
+  const beforeDelete = await requestEvents(relayWsUrl, [{ ids: [signedOriginal.id] }]);
+  assert.equal(beforeDelete.length, 1, "event should exist before deletion");
+
+  // Publish kind 5 deletion referencing the original
+  const deletion = {
+    kind: 5,
+    pubkey,
+    created_at: unixNow(),
+    tags: [["e", signedOriginal.id]],
+    content: "",
+  };
+  deletion.id = computeEventId(deletion);
+  const signedDeletion = await account.signer.signEvent(deletion);
+  const deleteFrame = await publishEvent(relayWsUrl, signedDeletion);
+  assert.equal(deleteFrame[2], true, `deletion publish failed: ${deleteFrame[3]}`);
+
+  // Query again -- the original should be gone
+  const afterDelete = await requestEvents(relayWsUrl, [{ ids: [signedOriginal.id] }]);
+  assert.equal(afterDelete.length, 0, "event should be deleted after kind 5 request");
+});

test/parrhesia/auth/challenges_test.exs

@@ -13,6 +13,7 @@ defmodule Parrhesia.Auth.ChallengesTest do
     assert Challenges.valid?(server, self(), challenge)
 
     refute Challenges.valid?(server, self(), "wrong")
+    refute Challenges.valid?(server, self(), challenge <> "x")
 
     assert :ok = Challenges.clear(server, self())
     assert Challenges.current(server, self()) == nil

test/parrhesia/plug_test.exs

@@ -0,0 +1,43 @@
+defmodule Parrhesia.PlugTest do
+  use ExUnit.Case, async: true
+
+  import Plug.Test
+
+  alias Parrhesia.Plug
+
+  test "init resolves configured listener id" do
+    opts = Plug.init(listener: :public)
+
+    assert is_list(opts)
+    assert Keyword.fetch!(opts, :listener).id == :public
+  end
+
+  test "init accepts inline listener map" do
+    opts =
+      Plug.init(
+        listener: %{
+          id: :host_mount,
+          features: %{nostr: %{enabled: true}}
+        }
+      )
+
+    assert Keyword.fetch!(opts, :listener).id == :host_mount
+  end
+
+  test "init raises for unknown listener id" do
+    assert_raise ArgumentError, ~r/listener :does_not_exist not found/, fn ->
+      Plug.init(listener: :does_not_exist)
+    end
+  end
+
+  test "call serves health route" do
+    opts = Plug.init(listener: :public)
+
+    response =
+      conn(:get, "/health")
+      |> Plug.call(opts)
+
+    assert response.status == 200
+    assert response.resp_body == "ok"
+  end
+end

test/parrhesia/protocol/event_validator_property_test.exs

@@ -0,0 +1,77 @@
+defmodule Parrhesia.Protocol.EventValidatorPropertyTest do
+  use ExUnit.Case, async: true
+  use ExUnitProperties
+
+  alias Parrhesia.Protocol.EventValidator
+
+  property "compute_id always returns lowercase 64-char hex" do
+    check all(event <- event_payload()) do
+      id = EventValidator.compute_id(event)
+
+      assert byte_size(id) == 64
+      assert {:ok, _decoded} = Base.decode16(id, case: :lower)
+    end
+  end
+
+  property "compute_id depends only on the canonical NIP-01 tuple fields" do
+    check all(
+            event <- event_payload(),
+            replacement_id <- hex64(),
+            replacement_sig <- hex128(),
+            extra_value <- StreamData.string(:alphanumeric)
+          ) do
+      original = EventValidator.compute_id(event)
+
+      mutated_event =
+        event
+        |> Map.put("id", replacement_id)
+        |> Map.put("sig", replacement_sig)
+        |> Map.put("extra_field", extra_value)
+
+      assert EventValidator.compute_id(mutated_event) == original
+    end
+  end
+
+  defp event_payload do
+    gen all(
+          pubkey <- hex64(),
+          created_at <- StreamData.non_negative_integer(),
+          kind <- StreamData.integer(0..65_535),
+          tags <- tags(),
+          content <- StreamData.string(:printable, max_length: 256),
+          sig <- hex128(),
+          id <- hex64()
+        ) do
+      %{
+        "id" => id,
+        "pubkey" => pubkey,
+        "created_at" => created_at,
+        "kind" => kind,
+        "tags" => tags,
+        "content" => content,
+        "sig" => sig
+      }
+    end
+  end
+
+  defp tags do
+    StreamData.list_of(tag(), max_length: 8)
+  end
+
+  defp tag do
+    StreamData.list_of(StreamData.string(:printable, min_length: 1, max_length: 32),
+      min_length: 1,
+      max_length: 4
+    )
+  end
+
+  defp hex64 do
+    StreamData.binary(length: 32)
+    |> StreamData.map(&Base.encode16(&1, case: :lower))
+  end
+
+  defp hex128 do
+    StreamData.binary(length: 64)
+    |> StreamData.map(&Base.encode16(&1, case: :lower))
+  end
+end

test/parrhesia/protocol/filter_property_test.exs

@@ -10,22 +10,83 @@ defmodule Parrhesia.Protocol.FilterPropertyTest do
             candidate_authors <- list_of(hex64(), min_length: 1, max_length: 5),
             created_at <- StreamData.non_negative_integer()
           ) do
-      event = %{
-        "pubkey" => author,
-        "kind" => 1,
-        "created_at" => created_at,
-        "tags" => [],
-        "content" => ""
-      }
-
+      event = base_event(author, created_at)
       filter = %{"authors" => candidate_authors}
 
       assert Filter.matches_filter?(event, filter) == author in candidate_authors
     end
   end
 
+  property "since and until filters follow timestamp boundaries" do
+    check all(
+            author <- hex64(),
+            created_at <- StreamData.non_negative_integer(),
+            since <- StreamData.non_negative_integer(),
+            until <- StreamData.non_negative_integer()
+          ) do
+      event = base_event(author, created_at)
+
+      assert Filter.matches_filter?(event, %{"since" => since}) == created_at >= since
+      assert Filter.matches_filter?(event, %{"until" => until}) == created_at <= until
+    end
+  end
+
+  property "tag filters match when any configured value is present on the event" do
+    check all(
+            author <- hex64(),
+            created_at <- StreamData.non_negative_integer(),
+            tag_value <- short_string(),
+            extra_values <- list_of(short_string(), min_length: 1, max_length: 5)
+          ) do
+      event =
+        base_event(author, created_at)
+        |> Map.put("tags", [["e", tag_value]])
+
+      matching_filter = %{"#e" => Enum.uniq([tag_value | extra_values])}
+      non_matching_filter = %{"#e" => Enum.map(extra_values, &("nomatch:" <> &1))}
+
+      assert Filter.matches_filter?(event, matching_filter)
+      refute Filter.matches_filter?(event, non_matching_filter)
+    end
+  end
+
+  property "invalid tag filters are rejected during matching" do
+    check all(
+            author <- hex64(),
+            created_at <- StreamData.non_negative_integer(),
+            invalid_value <- invalid_tag_filter_value()
+          ) do
+      event = base_event(author, created_at)
+
+      refute Filter.matches_filter?(event, %{"#e" => [invalid_value]})
+    end
+  end
+
+  defp base_event(author, created_at) do
+    %{
+      "pubkey" => author,
+      "kind" => 1,
+      "created_at" => created_at,
+      "tags" => [],
+      "content" => ""
+    }
+  end
+
   defp hex64 do
     StreamData.binary(length: 32)
     |> StreamData.map(&Base.encode16(&1, case: :lower))
   end
+
+  defp short_string do
+    StreamData.string(:alphanumeric, min_length: 1, max_length: 16)
+  end
+
+  defp invalid_tag_filter_value do
+    StreamData.one_of([
+      StreamData.integer(),
+      StreamData.boolean(),
+      StreamData.map_of(StreamData.string(:alphanumeric), StreamData.integer(), max_length: 2),
+      StreamData.list_of(StreamData.integer(), max_length: 2)
+    ])
+  end
 end

test/parrhesia/storage/adapters/memory/adapter_test.exs

@@ -6,6 +6,12 @@ defmodule Parrhesia.Storage.Adapters.Memory.AdapterTest do
   alias Parrhesia.Storage.Adapters.Memory.Events
   alias Parrhesia.Storage.Adapters.Memory.Groups
   alias Parrhesia.Storage.Adapters.Memory.Moderation
+  alias Parrhesia.Storage.Adapters.Memory.Store
+
+  setup do
+    start_supervised!(Store)
+    :ok
+  end
 
   test "memory adapter supports basic behavior contract operations" do
     event_id = String.duplicate("a", 64)

test/parrhesia/web/connection_test.exs

@@ -978,6 +978,41 @@ defmodule Parrhesia.Web.ConnectionTest do
     assert JSON.decode!(notice_payload) == ["NOTICE", message]
   end
 
+  test "websocket keepalive ping waits for matching pong" do
+    state =
+      connection_state(websocket_ping_interval_seconds: 30, websocket_pong_timeout_seconds: 5)
+
+    assert {:push, {:ping, payload}, ping_state} =
+             Connection.handle_info(:websocket_keepalive_ping, state)
+
+    assert is_binary(payload)
+    assert ping_state.websocket_awaiting_pong_payload == payload
+    assert is_reference(ping_state.websocket_keepalive_timeout_timer_ref)
+
+    assert {:ok, acknowledged_state} =
+             Connection.handle_control({payload, [opcode: :pong]}, ping_state)
+
+    assert acknowledged_state.websocket_awaiting_pong_payload == nil
+    assert acknowledged_state.websocket_keepalive_timeout_timer_ref == nil
+  end
+
+  test "websocket keepalive timeout closes the connection" do
+    state =
+      connection_state(websocket_ping_interval_seconds: 30, websocket_pong_timeout_seconds: 5)
+
+    assert {:push, {:ping, payload}, ping_state} =
+             Connection.handle_info(:websocket_keepalive_ping, state)
+
+    assert {:stop, :normal, {1001, "keepalive timeout"}, _timeout_state} =
+             Connection.handle_info({:websocket_keepalive_timeout, payload}, ping_state)
+  end
+
+  test "websocket keepalive can be disabled" do
+    state = connection_state(websocket_ping_interval_seconds: 0)
+
+    assert {:ok, ^state} = Connection.handle_info(:websocket_keepalive_ping, state)
+  end
+
   defp subscribed_connection_state(opts) do
     state = connection_state(opts)
     req_payload = JSON.encode!(["REQ", "sub-1", %{"kinds" => [1]}])
@@ -1004,6 +1039,8 @@ defmodule Parrhesia.Web.ConnectionTest do
       |> Keyword.put_new(:subscription_index, nil)
       |> Keyword.put_new(:trap_exit?, false)
       |> Keyword.put_new(:track_population?, false)
+      |> Keyword.put_new(:websocket_ping_interval_seconds, 0)
+      |> Keyword.put_new(:websocket_pong_timeout_seconds, 10)
 
     {:ok, state} = Connection.init(opts)
     state