a rfj[@sjdZddlZddlZddlZddlZddlmZdZdZ ddZ dd Z d d Z d d Z edkrfe dS)a Reconstruction of the "early-volume edge" finding, built fresh to verify it survives the same scrutiny that broke the paper quoter today. WHAT WENT WRONG TODAY (in mm_quoter.py / mm_settle.py, NOT this data): a taker/maker mapping that was briefly inverted, a raw-string timestamp comparison that silently filtered out every trade, and a fill test that counted IN-PLAY trades as if they were pre-kickoff resting fills. All three inflated results. This script re-derives the ORIGINAL historical finding from mm_trades.csv with none of those mechanisms present, and prints explicit sanity checks rather than asserting the number is right. METHOD: for every trade on a settled market, compute what SHARE of that market's total pre-kickoff volume had already traded before this trade (0.0 = first trade, 1.0 = last). Bucket into deciles. For each decile, compute the REALISED MAKER P&L using the verified taker_book_side mapping: taker_book_side == 'bid' -> taker BOUGHT -> MAKER SOLD at `price` taker_book_side == 'ask' -> taker SOLD -> MAKER BOUGHT at `price` This uses REAL trades that REALLY happened between REAL counterparties -- it is not a simulation of our own resting order, so it cannot inherit the fill-timing bugs that hit the paper quoter. It answers "what did the maker side of this market earn", not "would WE have gotten that fill" (queue priority is still unknowable from history -- unchanged caveat). SANITY CHECKS (printed, not assumed): 1. takers must be net BUYERS (~3-4:1) -- if not, the mapping is backwards. 2. total notional and event count should roughly match kalshi_backfill.csv's 641 events -- if far off, the join or filter is broken. Usage: python research\mm_decile_by_volume.py N) defaultdictz E:\soccer\research\mm_trades.csvz&E:\soccer\research\kalshi_backfill.csvcCsd|dddS)N-)joinsplittickerr )E:\soccer\research\mm_decile_by_volume.pyevent_of*sr c Cs8ztjt|ddWSttfy2YdS0dS)zParse to an aware datetime. Never compare timestamp strings directly -- that silently dropped every row in an earlier version of this analysis.Zz+00:00N)dtdatetimeZ fromisoformatstrreplace ValueError TypeError)sr r r pt.srcCsdi}ttddd>}t|D]"}t|d}|r|||d<qWdn1sV0Y|S)Nutf-8newlineencodingZkickoffZevent)openBACKFILLcsv DictReaderrget)outZfhrkr r r load_kickoffs7s,r"c Cst}ttttddd}dd|D}tt|ddt|dtd t|d td d |D}td d |D}td|dd|dd|t |ddd||krdndd g}d}}|D]}t |d} | | } | dur|d7}qt |d} | dus| | kr&|d7}qdt |dkrFdksJqq||qdd} |j| d |}td!|dtd"|dtd#t|dd$td%d|Dd&tt} |D]}| t |d|qtd'd(}tt}| D]\}}td)d |D}|dkr&qd*}|D]}t |d+}||}||7}t |dt|d,}}|d-d.kr||||}n||||}tt|d/d0}||d|7<||d||7<|||q.qtd1d2d3d4d5d3d6d7d3d8d9td/D]}||\}}|d:kr8qt||}|dkrRd;nd}t|d/d<d=|d/d/d<d>|d?d@|dAdB||dCdDdE| qtdFtdGt|ddHt|dIdS)JNrrrcSsh|]}t|dqSrr .0r r r r Dzmain..,z trade rows loaded, z distinct eventszkickoffs loaded for z eventscss&|]}|ddkrt|dVqdS)taker_book_sidebidcountNfloatr$r r r Ir'zmain..css&|]}|ddkrt|dVqdS)r)Zaskr+Nr,r$r r r r.Jr'zsanity: taker BOUGHT z,.0fz contracts, SOLD z -> buy:sell = z.1fz:1 (zOK, retail backs teamszSUSPICIOUS -- check mapping)rrtsg?Zpricegffffff?cSs4t|d}t|d|r|ntjjjtjjdfS)Nr1r)Ztzinfo)rr r rminrtimezoneZutc)r tr r r keyfks zmain..keyf)keyz dropped (no kickoff on record): z%dropped (in-play, at/after kickoff): z"PRE-KICKOFF, 5c-95c, TIME-SORTED: z rows, cSsh|]}t|dqSrr#r$r r r r&sr'z events cSsddgS)Nr r r r r }r'zmain..css|]}t|dVqdS)r+Nr,r$r r r r.r'r7r+Z result_yesr)r* ZdecileZ9s eventsz>7sZnotionalz>13sZROIz>8siz <-- opening tradesZ3drz% Z7dz $z12,.0fz dz+7.2f%zB sanity check 2: decile-0 event count should be close to the totalzK (every event contributes exactly one opening trade) -> decile-0 events = z vs z total events loaded)r"listrrrTRADESprintlensummaxr rrr-appendsortrsetitemsintr2addrange)ZkoZrowsr<Zbuy_ctZsell_ctZkeptZ dropped_no_koZdropped_inplayr er!r4r5Zby_eventZbucketZ bucket_eventsZevZtrstotalZcumcZ frac_beforepresZpnldZnotiZn_evflagr r r mainAs    "      &    L rS__main__)__doc__rZmathZ statisticsrr collectionsrr@rr rr"rS__name__r r r r s   _