--- title: Fix the three data-view generation corpus correctness defects type: task status: completed description: >- Three instructions in the 16-instruction generation benchmark corpus produce wrong results for three different reasons: a genuine multi-season agent defect, an unsatisfiable ground-truth assertion, and a misnamed season helper that makes the corpus shift across a season boundary. base_uri: user:task/league/data-views/fix-generation-corpus-correctness.md created_at: '2026-09-11T14:20:22.852Z' entity_id: c992d7c5-83bc-47be-9b9e-e6c28a2b73c9 finished_at: '2026-09-11T18:57:13.793Z' observations: - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ - ████████████████████████████████████████████████ owner_identity_uri: █████████████████████████ priority: Medium public_read: true tags: - user:tag/league-xo-football.md updated_at: '2026-11-06T02:27:59.666Z' visibility_analyzed_at: '2026-11-05T23:48:51.375Z' --- Three instructions in the 16-instruction data-view generation benchmark corpus do not produce right results. They surfaced together in the 2026-09-11 sweeps and they are three different KINDS of problem, so they need three different fixes. None of them is a latency question — [[user:task/league/reduce-data-view-generation-latency.md]] owns that and explicitly excludes these. ## Why this is its own lane The latency task ranks on output tokens and measures sweeps. These three are about whether the corpus tells the truth: one is an agent defect, one is a wrong assertion in the ground truth, and one is a helper whose name does not match what it computes. A correctness number read off a sweep is wrong until all three are settled, which is what makes them one lane rather than three tickets. ## The corpus and how to read it - Sweep runner and probe are documented in the latency task under "Running the benchmark". Run `scripts/data-view-benchmark-ground-truth.mjs --probe` FIRST, always — it exits 1 and names any unsatisfiable instruction, and a run against an unsatisfiable assertion fails in the exact shape of a wrong answer. - Sweep files live under `scratch/league/data-view-benchmark/`. The 2026-09-11 post-widening sweep is `post-grant-widening.jsonl`. ## Plan ### Fix the multi-season generation defect — `wr-receiving-yards-by-season-last-3` The only one of the three that is a genuine agent defect. Fails deterministically, both runs, 42.6s and 78.6s. Confirmed satisfiable by the probe, so the agent really is getting it wrong. It sits in the `row_axes` / `year_offset` multi-season family, which is what the generation contract carries its longest prose about — so the contract prose is a suspect, not just the code. It is newly exposed rather than a regression: it did not exist in the nine-instruction corpus, and those nine still score 18/18. Verify: the instruction goes correct across repeated runs, and the nine-instruction shared set does not regress. ### Correct the unsatisfiable assertion — `te-rookie-and-following-season-receiving-yards` NOT a generation defect. The ground-truth probe reports no `table_state` can satisfy it: the assertion expects a Brock Bowers row carrying both 1219 and 708, then disagrees with the database on the rank 2 through 5 ordering. Its 2-of-2 sweep failure and its failure in both A/B arms are the assertion being wrong. Decide what the instruction is actually asking, then make the assertion express it — or retire the instruction if it cannot be made to ask something coherent. Correcting for it alone moves real sweep correctness from 28/32 to 30/32. Verify: the probe exits 0 on the full corpus, and the instruction then passes or fails on its merits rather than by construction. ### Fix the clock-dependence — `last_completed_seasons()` and `wr-receiving-yards-last-3-seasons` The instruction went 2/2 correct to 0/2 with no code change between the sweeps. The emitted `table_state` is identical in shape and differs only in the year window: 2023-2025 before, 2024-2026 now. The cause is the calendar. Ground truth resolves its window with `last_completed_seasons()`, which despite the name just takes the last 3 season years carrying REG receiving plays — and 2026 week 1 landed between the two sweeps. So the reference window moved onto a one-week partial season, the agent tracked the move, and the ranking then disagrees on the partial year. The helper is misnamed against the instruction it serves, whose stated capability is a relative time expression that must resolve to COMPLETED seasons. Either make it mean what it says or rename it and fix the caller — but the corpus must stop shifting under itself across a season boundary. Verify: the sweep result for this instruction is stable across a simulated season-boundary crossing, not merely correct today. A result that is correct only at the instant you measured it is the defect. ## Assumptions - The three are independent and can be fixed in any order; nothing here assumes a shared root cause. - Deploys stay human-gated. Implement and verify, then present for ship.