---
name: Core Kalshi CLV edge significance — UNPROVEN and weakening (p 0.056→0.115 as n grew 365→394)
description: Live tracking edge ROI stays positive in every slice but is NOT statistically significant; significance moved the wrong way as data grew. Out-of-sample test confirmed the rolling-ROI gate adds nothing. Don't add rules; collect data.
type: project
---

As of 2026-06-06, the standing significance of the live Kalshi CLV strategy (home/away, band [5,12)%, bet at ask):
- n=365: one-sided z=+1.59, **p=0.056**.
- n=394 (29 more bets later same day's data refresh): z=+1.20, **p=0.115**. Significance moved the WRONG way — the new batch underperformed.
- ROI stays POSITIVE in every slice: full ~+18%, and a clean train/test split (TRAIN=first 60%, knobs locked, TEST=last 40%, n=146) gave TEST baseline **+22.9% ROI** (z=+1.19, p=0.117).

**Out-of-sample test of the rolling-ROI gate (added to strategy.ipynb 2026-06-06, 2 cells at end):** grid-searched gate knobs on TRAIN (best window=10/warmup=8/thresh=0.10, +74.4% in-sample ROI — looked amazing) then scored LOCKED knobs on untouched TEST: gated +12.9% ROI / p=0.31, i.e. WORSE than the +22.9% baseline. The in-sample gate gain was pure data-snooping. Gate is dead — don't revive it.

**Per-league breakdown (2026-06-06, full n=399, no exclusions, via `E:\soccer\_league_breakdown.py`):** User hypothesized the edge dropped after adding BSA, USL, Sweden, Norway. HALF WRONG: USL (-5.6u, ROI -26%) and BSA (-4.7u, ROI -36%) DO drag — but they're ALREADY excluded live (EXCLUDE_LEAGUES=['USL','BSA']). Sweden=**SE1 (+9.6u, ROI +88%)** and Norway=**NO1 (+3.5u, ROI +29%)** are among the BEST leagues — they HELP. With/without: ALL incl USL,BSA p=0.093; drop USL,BSA only p=0.056 (the live set); drop all 4 flagged p=0.062 (i.e. removing Sweden/Norway makes it WORSE). The biggest single non-excluded drag is **E0/EPL: n=23, 8.7% WR, -71% ROI, -16.3u** — but that's a cold streak on a core/most-tested market (auto-trade starts with E0), NOT a broken league; do NOT drop it (25% is natural WR; dropping red leagues by hindsight = the in-sample curve-fit the OOS test just punished). The existing USL/BSA exclusion is the correct and confirmed call.

**Why this matters / how to apply:** "Positive ROI but not significant" = UNPROVEN, not disproven. At n≈400 you cannot distinguish a small real edge (~+10%) from zero-edge-that-got-lucky-early-and-is-reverting; both produce this exact picture. The honest move is flat singles + conservative sizing (4.7% max DD insurance) + collect data; ~500-600 bets would get under p=0.05 IF the point estimate holds. The user got frustrated ("it's all bullshit") on seeing p=0.115 — the right framing is unproven/keep-collecting, NOT cheerleading and NOT capitulation. **Do NOT respond to a discouraging significance read by adding a filter/gate/sizing lever** — every such tweak has tested as noise (round robins, league gates, sub-bands, the gate). Adding leverage to an unestablished edge is how it turns into real losses. The discipline IS the strategy.