A daily snapshot of the pipeline that makes the forecasts: whether each scheduled job is running, how many forecasts and gradings it has produced, how much of the Supreme Court's docket it follows, and the results of its back-tests. The pipeline writes this snapshot itself every morning; this page shows it as written.
Is it running?
All 7 scheduled jobs are on time. The committed data passed validation.
A job is late when it has not started since its most recent scheduled time, allowing two hours because GitHub often starts scheduled jobs late. Only runs on the production branch count. A run that was skipped, or replaced in the queue by a newer one, counts as started but has no result.
Coverage
603events forecast241 of them now decided, 2,252 forecasts and 1,677 gradings in all
532pending petitions distributed for a conferenceas of Nov 11, 2026
167of those with at least one filing storedthe petition, a brief or another document the models can read
153petitions at the next conferenceNov 13, 2026
14,390dockets on the public ledger
2,154,093dockets in the research corpusthe history the base rates are read from
Back-tests
A back-test replays decided cases with the outcome hidden. Every outcome was public before the run, so a back-test measures recall of known history, not foresight. None of these figures is a claim about how well the models forecast; the results are.
Historical replay
metrics/backtest.json · Nov 9, 2026
2 simple baseline predictor(s) replayed over 58,039 resolved events in the corpus. No model runs here.
Retrospective: every outcome was public before the run, so this measures recall of known history, never foresight. It is an iteration instrument, and nothing here is a claimable performance figure. Read each lift only against its own row's floor. The pooled row mixes in courts this pipeline does not predict.
Cert back-test
metrics/cert-backtest.json · Oct 31, 2026
3 model(s) and 2 reference baseline(s) replayed over 10 decided cert petitions drawn at random from paid petitions, with each outcome hidden. Always-deny scores 90.0% on granted or not over this sample.
Retrospective: every outcome was public before the run, so this measures recall of known history, never foresight. It is an iteration instrument, and nothing here is a claimable performance figure. The floor is this sample's own denial share, so a different draw is scored against a different floor and is not comparable. No evaluator runs here, so a cell that saw its own outcome stays in and can only flatter its model.
Merits back-test
metrics/merits-backtest.json · Oct 25, 2026
3 model(s) replayed over 15 decided argued case(s) from OT2024, OT2025, at the grant and briefing moments, forecasting whether the judgment below is disturbed.
Retrospective: every outcome was public before the run, so this measures recall of known history, never foresight. It is an iteration instrument, and nothing here is a claimable performance figure. Read accuracy against the majority floor beside it. Brier skill is against a base rate that could have been known at each case's cutoff; a positive figure beats it. Cases are fenced or set aside in ways that can track their outcome, so every set here is a selected one.
Salience-gate replay
metrics/salience-replay.json · Oct 31, 2026
The gate that picks which petitions are predicted, replayed over OT2017, OT2018, OT2019, OT2020, OT2021, OT2022, OT2023, OT2024 under 3 cutoff policies.
Retrospective: every outcome was public before the run, so this measures recall of known history, never foresight. It is an iteration instrument, and nothing here is a claimable performance figure. It was produced under sal-v4 while sal-v8 is the scorer in force, so it describes that version and is not a current reading of the gate.