research
The Prediction Market Scorecard: A Live Brier-Score Track Record
By Odds Reference Published: Editorial Policy
The Prediction Market Scorecard: A Live Brier-Score Track Record
Published: November 2026 | OddsReference Research
The Scorecard is a continuously updated answer to one question: when prediction markets said something was 80% likely, did it happen 80% of the time? It scores every resolved contract we track across Polymarket, Kalshi, and Metaculus against what actually happened, and publishes the Brier scores, calibration curves, and upsets on the live scorecard.
What Does the Scorecard Measure?
The Scorecard measures whether market prices were honest probabilities. For every resolved contract it records the price a venue quoted at a fixed horizon before settlement and whether the outcome happened, then summarizes the gap with a Brier score, a skill score against the base rate, and a reliability curve.
Each number on the page is computed from the same underlying record: one row per contract, per venue, per horizon. A binary market contributes one contract for its YES side. A multi-outcome market, such as a 12-candidate nomination field, contributes one contract per candidate, each scored as its own binary question.
| Metric | What it tells you | Perfect | Uninformative |
|---|---|---|---|
| Brier score | Mean squared gap between price and result | 0.000 | 0.250 at a 50% base rate |
| Skill vs base rate | How much better than always quoting the base rate | +100% | 0% |
| Calibration error | Count-weighted distance from the diagonal, in probability points | 0% | Grows with over- or under-confidence |
| Log loss | Penalizes confident misses much harder than Brier | 0.000 | 0.693 at 50% |
The headline figure is the consensus Brier score one week before settlement. Consensus is the equal-weight mean of the venues quoting a contract, so a market listed on three venues counts once in the headline and once in each venue’s own row.
How Should You Read the Reliability Curve?
The reliability curve plots the price the market quoted against how often the outcome happened, in ten price bins. A perfectly calibrated market lies on the diagonal. Points above it mean outcomes happened more often than priced, so the market was under-confident; points below mean it was over-confident.
Dot size grows with the number of contracts in the bin, which matters for the low bins. Multi-outcome markets load the 0 to 10% bin with long-shot candidates, most of whom lose, so that bin is usually the best populated and the most reliable. The 40 to 60% bins are where genuine disagreement lives and where sample sizes are thinnest.
The curve is drawn for the consensus and for each venue, and the legend doubles as a toggle. A table view carries the same numbers for anyone who prefers them.
Why Does the Horizon Change the Answer?
Accuracy depends on when you ask. One day before settlement most markets have converged, and a Brier score at that horizon mostly measures whether the last trades were sane. Three months out the market is forecasting, not reporting, and the score is a fairer test of what the crowd knew in advance.
| Price taken | What the score mostly reflects |
|---|---|
| 1 day out | Convergence; whether late information reached the price |
| 1 week out | The headline horizon: enough lead time to be a forecast, enough data to be stable |
| 1 month out | Genuine foresight on events with a known date |
| 3 months out | Only long-dated markets qualify; smaller samples, harder questions |
Compare horizons against each other rather than against zero. A market that scores 0.08 one day out and 0.20 one month out is not contradicting itself; it is showing how fast information arrives.
How Are Venues Compared Fairly?
Venues are compared on shared contracts only. The head-to-head block scores two venues against each other using just the outcomes both of them priced at the same horizon, so the gap between them is about pricing rather than about which markets each venue chooses to list.
That restriction matters more than it sounds. A venue that lists mostly sports games, which resolve quickly and converge early, will post a better raw Brier score than one that lists long-dated political questions, without being any sharper. The per-venue table shows the raw numbers; the head-to-head block shows the fair ones. Our earlier prediction market accuracy report covers the same tension with a fixed snapshot of data.
How Do We Know What Actually Happened?
Results are inferred from each market’s final observed price and then verified against the venue’s own settlement record. A binary contract whose last price was at or above 0.97 counts as YES and at or below 0.03 as NO; a field needs a single outcome at 0.90 or higher with every other outcome at 0.10 or lower. Anything in between is held out rather than guessed.
The verification pass then asks the venue. Kalshi reports a result on every settled market. Polymarket’s UMA oracle pins resolved markets to exactly 1 and 0. Metaculus publishes a resolution on every closed question. Where the venue disagrees with the inference, the venue wins, and the correction is logged. The scorecard reports what share of its scored results have been venue-verified, and the methodology page states the rules in full.
Two limitations are stated rather than hidden. Markets whose final price never converged, typically because they went quiet before settlement, are excluded until a venue record settles them. And the first weeks of a new window lean more heavily on inferred results, with the verified share rising as the venue checks catch up.
What Counts as an Upset?
An upset is a contract the consensus priced at 85% or higher that resolved NO, or at 15% or lower that resolved YES, measured one week before settlement. The scorecard lists the largest misses with each venue’s price alongside the consensus, so you can see whether any single venue saw it coming.
Upsets are the most-read section and the least statistically important one. A well-calibrated market priced at 90% should lose one time in ten; a list of such losses is evidence of calibration, not of failure. The number to watch is whether the 90 to 100% bin on the reliability curve sits on the diagonal, which our calibration explainer walks through in more depth.
How Does This Differ From Our Earlier Accuracy Articles?
The earlier pieces were snapshots; this is a feed. Our guide to whether prediction markets are accurate and the 2026 accuracy report drew on fixed extracts of resolved markets. The Scorecard recomputes every six hours from the full resolved record of the last 365 days, so the numbers on the page are always current and always reproducible from the stated rules.
That also means the numbers move. A run of political upsets or a quiet sports month will shift the headline Brier score by a few thousandths. The history endpoint behind the page keeps every snapshot, so the direction of travel is visible, not just the current reading.
For the live treemap of every market feeding the Scorecard, see the Odds Reference dashboard.
Frequently Asked Questions
Which prediction market has the best Brier score?
It changes with the window and the horizon, which is why the Scorecard is live rather than a one-time report. The fair comparison is the head-to-head block, which scores two venues only on the contracts both of them priced. Comparing venues on different market sets mostly measures which markets each venue lists.
Why does the scorecard report several horizons instead of one accuracy number?
A price one day before settlement has almost all the information; a price three months out is answering a much harder question. One number would hide that. The Scorecard reports 1-day, 1-week, 1-month, and 3-month horizons so you can see how quickly the markets converge on the truth.
Can a Brier score be compared across categories?
Not directly. Categories with lopsided base rates, where most contracts resolve NO, produce low Brier scores even for an uninformative forecaster. The skill score corrects for this by comparing the market to a forecaster who always quotes the base rate, so compare skill across categories, not raw Brier.
How are market results determined when venues report them differently?
Results are first inferred from each market’s final price, then checked against the venue’s own settlement record: Kalshi’s settled result, Polymarket’s UMA resolution, and Metaculus’s resolution field. A confirmed result keeps its status, a wrong inference is corrected, and the share of venue-verified results is shown on the page.
Key Takeaways
- The Scorecard is a live, reproducible track record: every resolved contract in the last 365 days, scored at four horizons, recomputed every six hours.
- Read the consensus Brier score at one week as the headline, and the skill score when comparing categories with different base rates.
- Venue rankings are only fair on shared contracts; use the head-to-head block, not the raw per-venue table, to say who is sharper.
- A list of upsets is evidence that 90% means 90%, not evidence that the markets failed; check the top bin of the reliability curve instead.
- Results are inferred from final prices and then verified against venue settlement records, with the verified share published alongside the numbers.