--- title: Telemetry Series type: text description: >- How numeric telemetry rails become telemetry series in the stats database: declared sources, the rsync collector with idempotent byte-position loading, the operator-only /api/telemetry routes, and the /health page boards. base_uri: sys:system/text/telemetry-series.md created_at: '2026-11-04T21:18:29.965Z' entity_id: 41f03e6c-cc0e-4bac-bfc9-4bfe8524f269 owner_identity_uri: █████████████████████████ public_read: false updated_at: '2026-11-04T21:18:29.965Z' --- # Telemetry Series Telemetry rails (`telemetry//.jsonl`) are written on the host that measures, and never sync. A telemetry series is one numeric measurement from one machine under one label set, loaded from a rail into the stats database so the fleet's current state and history can be read in one place: the `/health` page. ## Store Three tables in the stats database (`libs-server/telemetry-series/telemetry-series-schema.mjs`), created by `base telemetry setup`: - `telemetry_series`: one row per series, with an integer `series_id`. Its `series_key` is a SHA-256 of name, machine and canonical labels, so a series keeps its identity whatever order or type a writer used for its labels. - `telemetry_samples`: `(series_id, observed_at, value)`. The integer key keeps a row near a quarter of the size a repeated hash would make it. Where the `timescaledb` extension is present it is a hypertable with 7-day chunks, compressed after 7 days. Elsewhere it is a plain table. Every query buckets with `date_bin`, so both work the same. - `telemetry_rail_positions`: how many bytes of each machine's rail day-file have been loaded. ## Loading `base telemetry collect` runs on one host, from cron. - For each machine in `telemetry_series.collect_machine_ids`, it rsyncs the day-files each declared rail still needs into `runtime/telemetry-rails///`. The path on the remote side comes from the machine registry's `user_base_directory`, and the local machine is copied the same way. - Each file is read from its stored byte position to the last complete line. Samples insert `ON CONFLICT DO NOTHING`, and the position advances in the same transaction. A crash, a database outage or a replay therefore writes nothing twice and loses nothing. A line still being appended is left for the next run. - The pull window starts at the newest loaded day, or three days back, whichever is earlier, so an outage of any length replays. A rail never loaded is pulled whole. - Mirror files are pruned once fully loaded and three days old. The host's rail stays the record. - `runtime/telemetry-rails/collect-telemetry-rails.json` is the oracle. A machine pull or rail load failing for 30 minutes raises `pipeline_failure:collect-telemetry-rails`, and a clean run clears it. - A series is filed under the registry slug it was collected from, not the row's own `machine_id`. ## Declaring rails and boards - Rails are declared by the `telemetry-series-source` capability, and by `libs-server/telemetry-series/core-telemetry-series-sources.mjs` for the rails core writes. - Boards are declared by the `telemetry-board` capability. - Both contracts are in `system/text/extension-system.md`. ## Reading `/api/telemetry` (`server/routes/telemetry.mjs`) is operator-only: it answers 401 to anonymous callers and 403 to any other identity. - `GET /series` returns bucketed history for one series name, optionally narrowed by machine and labels. The bucket widens so no series exceeds 2,000 points. - `GET /latest` returns the newest value per series within a week, with its age. A series older than three of its rail's cadences is stale. - `GET /catalog` returns series names with their machines and label sets, and every declared board. The `/health` page (`client/views/pages/HealthPage/`) renders each board and an Explore view over any series.