tribes-plugin-sender/docs/progress.md

Sender Implementation Progress

This file tracks the planned implementation from plugin template to clustered streaming plugin.

Phase 0: Architecture and Repo Preparation

• Capture planned architecture in docs/arch.md.
• Create this implementation checklist.
• Rename user-facing copy from generic "My Plugin" to Sender/Streaming.
• Update manifest.json description, capabilities, migrations, and children.
• Enable the plugin supervision tree in mix.exs.
• Decide initial media backend: supervised ffmpeg first, Membrane later behind a behaviour.
• Decide MVP control plane: Legion drives admin/orchestration through Tribes plugin management methods.
• Decide local HLS spool root config key: config :tribe_one_sender, TribeOne.TribesPlugin.Sender.HLS, spool_root: ....
• Document required runtime binaries in deployment packaging: at least ffmpeg.

Phase 1: Ash Domain and Synced Schema

• Add TribeOne.TribesPlugin.Sender.Streaming Ash domain.
• Register the domain through the plugin spec ash_domains.
• Add migrations for plugin tables.
• Add Stream resource with AshNostrSync.
• Add StreamKey resource with only hashed/verifier key material.
• Add StreamGeneration resource with AshNostrSync.
• Add MediaEndpoint resource with AshNostrSync.
• Model endpoint parent links for HLS distribution instead of a separate route graph.
• Add Rendition resource with AshNostrSync.
• Add EndpointSnapshot resource with AshNostrSync.
• Add a bounded EndpointHistory resource or defer it behind a JSON ring buffer.
• Add code interfaces for common actions instead of calling Ash.* directly from web modules.
• Add tests for resource creation, sync actions, and soft-delete behavior.

Phase 2: Single-Node RTMP to HLS MVP

• Add TribeOne.TribesPlugin.Sender.MediaBackend behaviour.
• Implement TribeOne.TribesPlugin.Sender.MediaBackend.FFmpeg.
• Add role-aware TribeOne.TribesPlugin.Sender.MediaSession runtime owner for origin_ingest.
• Run FFmpeg through MuonTrap instead of bare Erlang ports.
• Supervise ingest pipeline processes.
• Generate a stream key and validate RTMP ingest attempts.
• Declare Sender plugin management methods for Legion: capabilities, stream.*, stream_key.create, media_endpoints.upsert, renditions.upsert, and endpoint_snapshots.report.
• Add management handlers for default stream get/create/update.
• Add management handler for one-time stream key creation, marked sensitive.
• Add management handlers for endpoint, route, and rendition upserts.
• Add stream.start management handler with initial origin_ingest mode.
• Add stream.stop and stream.status management handlers.
• Start a StreamGeneration when Legion starts a stream.
• Write HLS output to spool_root/streams/:stream_id/:generation_id.
• Create initial Rendition rows from configured output variants.
• Keep a started generation active across source disconnects/backend exits until admin stop.
• Mark generation live once the first playable manifest exists.
• Mark generation ended on explicit admin stop.
• Mark generation failed when startup cannot create an initial backend session.
• Prune live edge spools and clean stopped generation spools according to retention config.
• Add tests around backend command construction and lifecycle state changes.

Phase 3: HLS Serving and Playback API

• Add plugin API route for playback metadata.
• Add plugin API route or plug for HLS playlist/segment serving.
• Enforce stream visibility before returning playback data.
• Add cache headers suitable for live HLS.
• Avoid caching future-segment 404 responses.
• Return generation-scoped URLs only.
• Add basic player LiveView page.
• Replace placeholder JS with a player bootstrap.
• Use TypeScript for the player bootstrap.
• Add Video.js v10 beta to the assets pipeline.
• Render playback with Video.js v10 HTML/custom-elements player.
• Report basic player errors and startup metrics to a plugin API endpoint.
• Add page/API tests for playback authorization and not-found behavior.

Phase 4: Local Stats and Snapshots

• Track local active viewers by endpoint/generation.
• Track ingress/egress bitrate estimates.
• Track playlist age and last segment timestamp.
• Publish EndpointSnapshot rows from the local node only.
• Treat stale local stats as degraded before publishing.
• Add local telemetry events for ingest, segment generation, playback, and errors.
• Add a periodic snapshot worker.
• Add tests for snapshot staleness and ownership rules.
• Align monitoring plan with Tribes' host metrics subsystem: /metrics, node-local VictoriaMetrics, synced rollups, and plugin metric declarations.
• Declare initial Sender plugin viewer_count metric and map it into host-exported plugin metrics.
• Add compact rollups for ingress/egress bitrate, playlist age, and segment lag when those local samples are mature enough.

Phase 5: Cluster-Aware HLS Distribution

• Accept Legion-authored endpoint source links through management methods.
• Fill edge-node local HLS spools from each endpoint's configured source endpoint.
• Serve HLS from /sender/hls/ through Vinyl.
• Keep viewer playback on the current node that served the player/API.
• Add configurable health thresholds.
• Add tests for endpoint source-link behavior under healthy, degraded, stale, and overloaded endpoints.
• Make every Tribes node able to answer playback metadata for any synced live stream.

Phase 6: Pull-Through HLS Edge Mode

• Add endpoint type tribes_edge/external_edge.
• Implement pull-through fetch from an upstream HLS origin.
• Cache finalized segments locally.
• Keep live playlist caching conservative.
• Add upstream failover based on endpoint source links.
• Emit cache headers compatible with Vinyl.
• Verify regular HLS origin/edge behavior before enabling LL-HLS edge behavior.
• Add tests for cache hit, miss, upstream failure, and stale manifest handling.

Phase 7: External Mux and Fanout Nodes

• Model external mux nodes as MediaEndpoint(type: :external_origin).
• Model external HLS repeaters as MediaEndpoint(type: :external_edge).
• Accept Legion-reported external endpoint snapshots through management API.
• Document Legion-owned lifecycle for external media processes.
• Add signed/token-authenticated stats ingestion API.
• Add optional polling for external HLS origin health.
• Validate external endpoint payloads before writing synced snapshots.
• Support multiple renditions produced by an external mux.
• Support endpoint source links where external mux fans out to Tribes nodes and external edge nodes.
• Add tests for external stats auth and endpoint source-link selection.

Phase 8: LL-HLS

• Switch or add ffmpeg output mode for CMAF/fMP4 segments.
• Add LL-HLS playlist settings and target part duration.
• Handle partial segment serving.
• Review preload hint and blocking reload behavior.
• Tighten cache headers for parts and playlists.
• Add player low-latency config.
• Add metrics for live edge distance and rebuffering.
• Test Safari native LL-HLS behavior separately from Video.js' HLS engine path.

Phase 9: Membrane Evaluation

• Identify Membrane packages needed for RTMP ingest, HLS output, and optional transcoding.
• Build a backend prototype behind TribeOne.TribesPlugin.Sender.MediaBackend.
• Compare operational behavior against ffmpeg: startup, failure handling, CPU, memory, latency.
• Decide whether Membrane becomes primary, optional, or deferred.
• Keep ffmpeg backend available unless Membrane fully covers production needs.

Phase 10: Packaging and Operations

• Add runtime config for ports, spool root, retention, backend, and route thresholds.
• Add Nix deployment packaging for ffmpeg.
• Add Nix packaging for Sender as a packaged Tribes plugin.
• Add single-image Docker E2E packaging for a two-node origin/edge scenario.
• Add spool directory creation and permission checks on startup.
• Add health checks for required binaries and writable spool.
• Add optional standalone Sender admin UI for stream setup, endpoint status, and current live sessions.
• Add runbook docs for OBS configuration.
• Add runbook docs for Legion-driven single-node deployment.
• Add runbook docs for multi-node HLS edge deployment.
• Add runbook docs for external mux/fanout deployment.

External Dependencies

Required for the first useful MVP:

• ffmpeg runtime binary in Nix/Docker packaging.
• Video.js v10 beta HTML player package, @videojs/html.
• Writable local spool directory.
• Legion client support for Sender plugin management methods.

Already available through the Tribes host or current plugin setup:

• Phoenix/Plug routing for plugin pages and API routes.
• Tribes plugin management methods over /api/admin/management.
• Ash and AshPostgres for persisted resources.
• AshNostrSync for cluster-replicated plugin state.
• Telemetry primitives.
• Tribes metrics subsystem: /metrics, VictoriaMetrics client, rollup worker, synced rollups, admin API, and plugin metrics contract.

Optional or future:

• Membrane packages for BEAM-native media routing.
• MuonTrap or similar for stricter external process supervision.
• Vinyl in deployment for HLS caching.
• Declare Vinyl authoritative viewer and playlist rollups for proxied/cached playback.
• External mux/transcode service for multi-bitrate production.