--- name: project-season-rollover-checklist description: What breaks at every European season rollover and how to fix it — misfiled football-data CSVs, fdco RENAMES of clubs that never moved, stale div fields, and unmapped promotions. Includes the 2026/27 pass and the three scripts that automate it. metadata: type: project --- Done 2026-08-22 for the 2026/27 rollover. Four distinct failure modes, all SILENT, all found in one pass. Re-run this every August. ## 1. football-data.co.uk serves the WRONG FILE (worst one) `E0_2627.csv` held EC (National League) rows; `SP1_2627` held P1; `SP2_2627` held SC2 — byte-identical duplicates of the other league's file. These CSVs are the last-resort source for bet resolution, so wrong rows are worse than missing ones. Cause is upstream content negotiation: `https://.../mmz4281/2627/E0.csv` answers **HTTP 300 Multiple Choices**, and depending on headers can serve a neighbouring file. `refresh_league_data.py` only checked `size < 200`, which a complete-but- wrong CSV passes trivially — five days undetected. FIXED: the downloader now reads the `Div` column every row carries, refuses any file whose content does not match the requested division, and writes via a temp file so a bad fetch cannot overwrite a good one. Run `research/audit_league_files.py` to check all 170 files (it uses **utf-8-sig** — some ship a BOM, and plain utf-8 makes `Div` lookups silently return None, which reported 60 files as "empty" on the first attempt). STILL BROKEN UPSTREAM: `E0, D1, I1, I2, F1, G1` return HTTP 300 and have NO 2627 file at all. That is the Premier League, Bundesliga, Serie A/B, Ligue 1 and Greece. Retry periodically; `E0_2627.csv.mismatch-contains-EC` is the quarantined bad copy. ## 2. fdco RENAMES clubs that never moved This is the sneaky one — the club is still in the same division, so nothing looks like it changed, but the `fdco` field no longer matches the results CSV and resolution via that fallback stops finding the fixture. 2026/27: `Ath Madrid`→`Atl. Madrid`, `Vallecano`→`Rayo Vallecano`, `La Coruna`→`Dep. A Coruna`, `Bradford`→`Bradford City`. Also seen: E2_2627 contains **both** `Sheffield Wed` and `Sheffield Weds`, one row each. Do NOT add a second entry for the same club — the resolver's exact-then-substring match covers it. ## 3. Stale `div` on promoted/relegated clubs 20 entries this year. Mechanical once found. ## 4. Unmapped promotions 19 new entries. **NEVER GUESS THE KALSHI ABBREVIATION** — the 2026-05-19 audit caught three wrong guesses, and a wrong abbr fails silently as a permanent `Kalshi=?A/?H` ([[feedback-team-map-silent-miss]]). `research/harvest_kalshi_abbrs.py` pulls verified abbreviations from real event tickers: every market's `rules_primary` names both clubs in full ("... the Kilmarnock vs Dundee United professional ..."), and the per-outcome ticker suffix is that club's abbr. Two traps in it, both fixed: the rules regex needs a GREEDY `.*` prefix or it binds to the first "the" and captures "result of the Kilmarnock"; and the exchange rate-limits after ~4 series, which a silent `break` on non-200 turned into a convincing "0 abbreviations" for half the divisions. team_map is KEYED BY THE API-FOOTBALL NAME, not the fdco one. Source the AF spelling from watchlist.txt / live_data / raw_api rather than inventing it — several differ sharply (`ST Johnstone`, `Çorum FK`, `Celta de Vigo II`, `Lommel United`). ## Still open after this pass - `Willem II` (N1): Kalshi lists no market, so no abbr exists to harvest. Entry added with fdco only; fill in `kalshi` when they appear. - The six divisions with no 2627 file (see §1). ## Scripts research\audit_league_files.py every CSV's Div vs its filename research\find_promoted_teams.py movers + what is unmapped or stale research\harvest_kalshi_abbrs.py verified abbreviations from the exchange research\fix_team_map_2627.py applies the corrections (dry run by default) team_map.json hot-reloads — `live_monitor` picks changes up within a poll cycle, no restart needed. Backups are written as `team_map.json.bak_`.