--- title: Reduce data-view generation latency below 30 seconds type: task status: in_progress description: >- Get agentic data-view generation reliably under 30s per instruction. Measured: 98 percent of a run is inference time and 86 percent of that is decode, so output tokens drive duration (r2 0.46) and turn count does not (r2 0.06); engine concurrency swings it 5x on identical work. base_uri: user:task/league/reduce-data-view-generation-latency.md created_at: '2026-09-04T18:43:43.837Z' entity_id: a1a50ac7-a22f-43a5-935c-4535176a747c observations: - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ owner_identity_uri: █████████████████████████ priority: Medium public_read: true tags: - user:tag/league-xo-football.md updated_at: '2026-10-25T00:33:17.282Z' visibility_analyzed_at: '2026-11-05T23:49:57.889Z' --- ## See it live https://xo.football/data-views Get agentic data-view generation reliably under 30 seconds per instruction, measured by the 9-instruction benchmark in `repository/active/league`. **Status 2026-09-10: not met, and the cost model the task opened with was wrong.** Across 129 completed non-benchmark jobs the distribution is p50 53.5s, p95 448.5s, max 707.8s — the objective is missed by the MEDIAN run, not by a tail. ## What actually costs the time Measured over 268 completed generations, joining `data_view_generation_jobs.thread_id` against `telemetry/inference-gateway/2026-09-*.jsonl` on ████████████: | term | share / score | | --- | --- | | wall clock spent inside inference requests | 98% | | of request time: decode (token generation) | 85.6% | | of request time: prefill (time to first content token) | 14.4% | | output tokens vs duration | r2 0.460 | | request count vs duration | r2 0.353 | | turn count vs duration | r2 0.056 | **Rank on OUTPUT TOKENS, not turns and not wall clock.** Under-30s runs write ~2,681 output tokens, over-120s runs ~9,308. Queue, dispatch and tools are ~2% of a run. **Engine concurrency is the largest single factor and the agent cannot touch it.** Decode on `vllm-deepseek-v4-flash` falls 209 tok/s at one in-flight request to 30 at twelve. At job level with output volume flat: 38.4s at concurrency 1 (4,255 out tokens) against 196.0s at concurrency 8 (4,379). Under-30s runs sit at median concurrency 1.0, over-120s at 5.0. League generation is identity-gated to this engine and cannot take the capacity divert. ## Two distinct problems **The tail is betting-market questions** — 9 of 263 runs, median 265s against 59.3s, 30 tool calls against 9. Root cause found and fixed: `describe_column` resolved param keys against the shared registry only and dropped any the column defined itself, silently, across 314 columns and 502 param slots. The betting family lost `market_type`, `source_id`, `selection_type` and `time_type` — so a prop column could not be configured from the tools at all and the agent fell through to `run_sql` and league source. `run_sql` is the sharpest marker of a bad run anywhere in the corpus: 20 runs used it at median 362.3s against 56.3s. **The median is registry-only questions** — 254 runs at median 59.3s and 9 tool calls, of which 64% precede the first `validate_table_state`. Shape: contract Read, 2-4 catalog calls, validate, preview, emit. ## Done and deployed (production `ca3c80ef0`) - **`describe_column` param drop** — league `d72263341`. The column's own `param_overrides` now wins, and a key nothing can define is reported as `params_without_definition` rather than omitted. Zero gaps across all 598 columns, held by a spec. - **Cost model corrected** in both the contract (`generate-data-view.mjs`) and the cached profile prompt (`cli/league/build-data-view-generation-profile.mjs`) — charges for output volume, not turn count. - **emit / preview_view ambiguity** — league `ef70e542f`. A `candidate_id` alongside an identical `table_state` is accepted; only a real disagreement refuses. Hit 24 of 274 runs at the most expensive point of a run. - **Redis stderr noise** silent under `NODE_ENV=sandbox`; it reached 20 runs and `--no-warnings` does not suppress it. - **Contract additions** — the five identity ids named as exact and param-free, and a starter card of the most-wanted columns, guarded by a spec asserting every column id it names exists in the catalog. - Earlier: the misleading validator error (`76c2855a1`), the contract's own trap-teaching example (`232c1aa34`), two benchmark-runner defects (`597712e57`). **Measured 2026-09-11** (`scratch/league/data-view-benchmark/post-grant-widening.jsonl`): the mechanism landed and the corpus median did not move. `permission denied for materialized view` went 6 threads to 0, and the targeted prop instruction halved (8,511 to 4,225 output tokens), while the sweep median stayed flat (2,058 to 2,190). Per-instruction ratios of 0.77x to 1.61x at n=2 are the documented spread, not signal. Two things shipped since and are NOT in that sweep: the `run_sql` audit fix (league `711624043`, deployed) and the source-diving posture change (league `922ff2ee8`, propagating by sync). Neither has been measured. ## Closed levers — do not reopen - **Contract-in-prompt**: the contract-FILE-READ profile wins (18/18 correct, 7.25 paired turns vs 7.57, build calls 3.29 vs 5.93 inline). Do not re-inline. - **Harness comparison**: negative — a `pi` session never emits. - **The prefill blind spot**: a non-issue. The gateway records cache token counts null on every generation request (`usage_source` is `anthropic_stream`), and `time_to_first_content_token_ms` already measures prefill. - **The 30s objective and the ████████████████████ bands**: operator ruled them unchanged; a move to 45s was declined. - **Backfilling the 12 malformed stored job results**: operator ruled against it. - **Effort level**: evaluated and showed no benefit; operator ruled it closed 2026-09-11. The `deepseek-v4-flash-xhigh` entry's recorded ~34% output-token reduction did not carry to this workload. ## Plan The remaining levers, in the order they are worth taking. - [ ] **Source-diving.** 69 of 263 runs (26%) read league source despite the prompt and contract both forbidding it, median 78.5s against 51.5s. Prohibition failed across three revisions, and the operator ruled 2026-09-11 that source-diving is sometimes needed, so the target is a sanctioned narrow path, not a fourth prohibition. The design question is which surfaces justify it — the ones agents reach for are exactly the ones the tools do not answer, so each recurring dive is either a missing tool capability or a legitimate edge case, and separating those two is the work. - [ ] **Engine contention.** Getting the median under 30s at concurrency 5 needs capacity reservation or a dedicated engine — an infrastructure decision, arguably not this task's. ## Measurement hazards, all paid for - The generation profile has `max_concurrent_threads: 1`. Check `base thread list --running | grep -E 'data-view-generation-[0-9a-f]{8}'` before any sweep; a held slot 429s on create-session. - Record ████████████'s league SHA at sweep start or the comparison is confounded (the sync loop advanced the submodule mid-sweep on 2026-09-04). - Scope every query `principal_key NOT LIKE 'benchmark:%'` — 145 of 274 completed jobs are benchmark rows. `user:1`, `user:130`, `user:134`, `user:138` are all real. - Exclude `dispatched_at: null` and `provider_error` rows before scoring. A null-turns completed row is absence of data, not zero. - `baseline-9.jsonl` is NOT a measurement (three runs never dispatched) — never diff against it, and never REGRESS on it either: including it moves turns-vs-duration from r2 0.056 to 0.705 on one high-leverage point. - A run reaping with "Thread not found" and no transcript metrics is the known rsync-window race — retry the read once. - Classify tool calls with `(?:data|plays)-view-([a-z_-]+)\.mjs`; a `data-view-` prefix alone misfiles every plays_view run as ad-hoc shell. - A single-sitting A/B on prefix_cache_delta measures ARM ORDER, not the lever. Run both orders or report no comparison. - A job can queue behind dead dispatches: `await_repl_ready` blocks the worker its full 300s on a REPL whose policy layer never reports (job `b901d33b` waited about five minutes, then ran in 14s). Subtract queue time before calling a run slow. - The league tree is shared. Never build or deploy from it — `preflight:deploy` prints the throwaway-worktree recipe; install with `--immutable`, never symlink the shared `node_modules`. ## Running the benchmark ``` LEAGUE_DB_PORT=15432 NODE_ENV=production node scripts/data-view-benchmark-run.mjs --repeat 2 --out NODE_ENV=production node scripts/data-view-benchmark-ground-truth.mjs --probe ``` Roughly 30-60 minutes for `--repeat 2`, serialized because the profile permits one concurrent session. Output under `scratch/league/data-view-benchmark/`; add a line to that directory's ABOUT.md naming the sweep. Run `--probe` after touching any instruction. As of 2026-09-30 the `refuse_stale_assertions` preflight aborts every sweep: re-derived ground truth for `qb-passing-yards-prop-ladder-2024-week-1` has a null measure for `BAKE-MAYF-002054`. Re-derive it or exclude the row first. ## Cluster ███████████████, 2026-09-23 Four `████████████████████` rows triaged onto this task. Every one verified live-principal by reading `data_view_generation_jobs` directly — none is a benchmark row. | Signal | Band | Elapsed | Calls | Output tokens | Principal | Generation | | ------ | ---- | ------- | ----- | ------------- | --------- | ---------- | | 131164 | severe | 426s | 66 | 120,393 | `user:1` | `128821b7` | | 131067 | slow | 166s | 16 | 73,425 | `user:1` | `29741699` | | 131291 | over_target | 89s | 9 | 27,199 | `user:130` | `f29e6211` | | 131182 | over_target (plays_view) | 75s | 4 | 15,759 | `user:130` | `b7069083` | All four ran on `vllm-deepseek-v4-flash`. **131164 is the runaway-scale agentic loop, not a query or contention defect.** One instruction — team defense series success rate allowed, drive-touchdown percentage, explosive run rate allowed — drew 66 tool calls and 120K output tokens against the 900s deadline. Same class as 130507, 130654 and 129589, and the per-job spend ceiling this task owns is what bounds it. The other three sit inside the established post-JIT-off floor and need nothing of their own. **The payload now names the principal.** `report-slow-generation.mjs` omitted `principal_key`, so every triage pass had to re-join `data_view_generation_jobs` to tell a benchmark run from a real one; a null `inference_provider` is NOT a discriminator, since older genuine rows carry it too. League `ce44be5b8` carries `principal_key` in the payload, with a spec proven to fail against the old source. The `NOT LIKE 'benchmark:%'` hazard above still governs queries against the jobs table; it is now also answerable straight from the signal.