a >wj\@sdZddlZddlZddlZddlZddlZddlmZddlm Z ej ddej ddddl Z e dZe dZd Zd d Zd d ZddZedkredS)aWhich Pinnacle feed is more ACCURATE: pinnapi's live line or API-Football's relay? WHY THIS TEST AND NOT THE OBVIOUS ONE. The instinctive test is "do pinnapi's signals win more bets". Measured 2026-08-08, that needs ~3,100 scored signals -- about eight months -- because the signals the two feeds disagree on sit at ~0.11 implied, where per-bet ROI stdev is ~2.84 units. This instead scores each feed's devigged fair value against the realised result with a Brier score, PAIRED per fixture. Two properties make it ~50x cheaper: 1. It uses EVERY fixture (~100-200 resolve daily), not just the ~13/day where the signals disagree. 2. The feeds agree most of the time, so the paired DIFFERENCE has far less variance than either feed's raw accuracy. Measured stdev 0.1186, so ~580 fixtures resolves a 0.01 effect -- roughly a week. WHAT A POSITIVE RESULT DOES AND DOES NOT MEAN. `fresh_better > 0` means pinnapi's line predicts outcomes better. It does NOT mean switching the feed makes money: pinnapi shifts xlow's signals toward ~0.11 long shots, and the 2026-05-23 audit found the rule bled there (hence EDGE_UPPER_CAP at 12%). A win here justifies RECALIBRATING the band against the fresh line and re-testing -- not flipping the live path, which stays on API-Football so the frozen forward test keeps running. Results are cached in research/pin_accuracy_cache.json so repeated runs do not re-query Kalshi for fixtures already settled. Usage: python research\pin_accuracy_test.py [--target N] N) defaultdict)PathzE:\soccer\mainzE:\soccer\researchz(E:\soccer\research\pin_fresh_capture.csvz*E:\soccer\research\pin_accuracy_cache.jsoniDc Cs(z t|WSttfy"YdS0dS)N)float TypeError ValueErrorxr 'E:\soccer\research\pin_accuracy_test.py_f/s r cCs0zttjddWSty*iYS0dS)Nutf-8encoding)jsonloadsCACHE read_text Exceptionr r r r load_cache6s rc# Csft}dtjvrDzttjtjdd}WnttfyBYn0tsZt dti}t tddn}t |D]R}|dd|dd|d d|d }||vs|d ||d krv|||<qvWdn1s0Yt }gttd }}}d } } |D]0\}}t|dt|d} } t|dt|d} }d| | | |fvrfq||}|dur|d\}}}}t||||d\}}|r|||<|s|d7}qd|dkd|dk}}| |d| |d}| |d||d}| |7} | |7} |||||d||qztjtj|ddddWntyvYn0t|}tdtd|d|dtd||dkrtddSt|}t |pd}|t!"|}|r||nd }td | |d!td"| |d!td#|d$d%|d!d&|d'd|d(||d(|}}td)|d$d*|d$d+t||krd(|t#t$|d,d} td-||d.| d/n4t$|d(krtd0n|d krtd1ntd2ttd3t%|d4d5d6dd7D]B\}!}"t|"d8krtd9|!d:d;t|"d<d=t|"d$qdS)>Nz--targetzno capture at r r lc|homeawayZ game_dateZtsrgZ fair_fresh_hZ fair_fresh_aZ fair_stale_hZ fair_stale_a)Zdivrrdateg?HA)indentz7PINNACLE FEED ACCURACY -- paired Brier, lower is betterz fixtures resolved : z (target )z still unresolved : z too few to reportg& .>z mean Brier (pinnapi) : z.5fz mean Brier (API-Ftbl): z paired difference : z+.5fz (SE z, z z+.2fg\(\?z 95% CI : [z, ]g-C6?z NOT YET CONCLUSIVE -- z7 more fixtures to target; observed effect would need n=z,.0fz} VERDICT: no significant accuracy difference at n>=target. The relay is good enough; staleness is not distorting fair value.z VERDICT: pinnapi is significantly MORE accurate. Next step is to RECALIBRATE the xlow band against the fresh line and re-test -- not to switch the live feed, which would break the frozen series.zE VERDICT: pinnapi is significantly LESS accurate. Keep API-Football.z3 by league (paired difference, + favours pinnapi):cSst|d S)Nr)lenrr r r zmain..)key z Z6sz n=Z4dz )&TARGETsysargvintindex IndexErrorrSRCexists SystemExitopencsvZ DictReaderrrlistitemsr getsplitDSZfetch_result_kalshiappendrZ write_textrdumpsOSErrorr"print statisticsZmeanZpstdevmathZsqrtmaxabssorted)#targetZlastZfhrkcacheZdiffsZby_lgZ unresolvedZ fresh_totZ stale_totZfh_Zfa_Zsh_Zsa_Zftrrrrgd_ZyhZyaZbfZbsnmZsdZsezlohiZneedZlgZdsr r r main=s *(     "     $rL__main__)__doc__r2rr=r<r) collectionsrpathlibrpathinsertZ daily_signalsr7r.rr(r rrL__name__r r r r s$  Y