Steam Launch Forecaster

Quarterly report · 2026-Q2 · Published 2026-05-09 · 12 min read

The Calibration Gap

Why most Steam revenue forecasts don't show you how wrong they are.

TL;DR

The active Steam Launch Forecaster model achieves 82% empirical coverage of its P10–P90 cones on a held-out validation set of 1,560 historical Steam launches — games that were specifically excluded from training and graded against realized week-1 revenue. Target was 80%. The held-out result is the methodologically pristine number: model fit, then evaluated on data it never saw, with realized outcomes from an independent revenue pipeline. We also publish a continuously-refreshed cohort verification (below) that runs the model daily on every game in our corpus that launched in a rolling 180-day window, and reports per-genre coverage and Wilson 95% CIs. Meanwhile, every commercial Steam revenue tool we surveyed either: (a) doesn't publish accuracy at all, (b) publishes only a single aggregate number, or (c) is a rule-of-thumb whose own author has admitted ~24% of games are off by more than 30%. That asymmetry — between what indie devs are betting hiring decisions on and what they're being shown about how often the bet is wrong — is the calibration gap.

What "calibration" actually means

If a forecast tool tells you "there's an 80% chance this game makes between $50K and $500K in its first week", calibration is the simple, testable claim that — across many such forecasts — the realized revenue lands inside the predicted range about 80% of the time. Not 50%, not 95%. Specifically 80%, give or take noise.

This sounds tautological until you notice that nobody actually checks. A forecast tool that says "80% chance" but is right only 40% of the time on the historical record isn't being conservative or being precise — it's being wrong about its own confidence. That's worse than no forecast at all, because the false confidence pushes downstream decisions (hiring, port commitments, marketing budgets) toward riskier bets than the data supports.

Statistical calibration is testable on a held-out set of past launches. You take forecasts the model would have made before launch, compare them against what actually happened, and count what fraction landed inside the predicted cone. If you don't see that number, you're not getting a forecast — you're getting a confident vibe.

The three camps of "Steam revenue prediction"

Across the indie tooling landscape there are three forecasting cultures. Each is testable in principle but only one of them shows you the test results.

Camp 1 — Rules of thumb

The dominant entry-level approach is Boxleiter's rule: revenue ≈ review_count × multiplier × price, where the modern multiplier sits between 30 and 63 sales per Steam review depending on genre and cohort. It's straightforward, easy to apply, and — importantly — its accuracy is publicly testable.

Mike Boxleiter himself, who has been the most thoughtful steward of this rule, has openly stated that roughly a quarter of games are off by more than 30% from a single-multiplier estimate. Most readers nod at this without internalizing the implication: 24% of the time, a one-shot Boxleiter prediction is wrong by a margin large enough to flip a hiring decision. The rule is honest about its own limits, which puts it ahead of most of the field — but a single-number estimate with a known one-in-four miss-by-30%-rate is not a calibration; it's a sanity check.

Camp 2 — Closed-source dashboards

The next tier up is the commercial-dashboard cohort: tools like VG Insights and several other commercial dashboards, all of which expose Steam-derived revenue estimates through paid SaaS portals. These tools clearly do statistical work behind the scenes — they fit models, refresh them, publish per-game point estimates, and charge serious money.

What they do not publish is the empirical accuracy of those estimates. We could not find, for any of the major commercial dashboards, a public per-genre coverage table, a held-out validation result, or even an aggregate "we are right ±X% of the time" disclosure. That doesn't mean their estimates are wrong — it means the accuracy is, by commercial choice, opaque to the buyer. If you pay for a forecast that doesn't disclose its miscoverage rate, what you're buying is the implication of accuracy, not a measurement of it.

Camp 3 — Vibes from successful devs

The third culture is the most influential: heuristics that propagate through indie-dev podcasts and Twitter threads. "Budget 10× your Kickstarter goal." "Most games make their lifetime revenue in the first month." "If you have under 7,000 wishlists, delay." These rules are useful as starting frames but are essentially untestable — they don't define the cohort, the time window, or the success criterion concretely enough to be falsified. They survive because they sound right to people who already succeeded, not because anybody measured them.

Calibration evidence — our data

Held-out validation (the gate number)

The active model on steamforecast.app was trained on a corpus of historical Steam launches with derived revenue outcomes. Of that corpus, 1,560 launches were held out from training: the model never saw their features during fit and was scored cold on them at the end. Aggregate empirical coverage of the published 80% cones on that held-out set: 82.0%.

The held-out number is the gate. A model that fails it doesn't ship; the prior model stays active until the new one passes. That's why this number is not a marketing claim — it's what determines whether code gets deployed.

It's also a small enough cohort that an honest table would show wider per-stratum confidence intervals than most commercial dashboards quietly carry. We expose those on /methodology with the corresponding training-set size and the per-quartile gate result.

Cohort verification (the ongoing test)

Held-out validation is a one-time snapshot. Cohort verification is the continuous companion: every day we take all games in our corpus whose release_date falls in a rolling 180-day window through 7 days ago (allowing a week for revenue to settle), run the active model on each one's pre-launch features, and compare to actual week-1 revenue. The cohort is defined by the launching-games population, not by who happens to call our endpoint — which means bot crawlers, AI agents, user lookups, and our own smoke tests all have zero influence on the number.

The cohort verification table renders inline on this page once the daily run has populated. While it bootstraps, the held-out 82% / n=1,560 above stands as the published calibration claim. The same data feeds /methodology as soon as it's available.

A methodology note about a prior version of this report

An earlier version of this page (published 2026-05-09) led with a much larger headline number: "88.4% live coverage on 11,921 logged forecasts." That number measured the model's coverage on whatever appids had been queried via /forecast?appid=…, whether by humans, AI crawlers, or our own smoke pings. Forensic investigation surfaced that ~99% of the 11,921 rows came from a single OpenAI GPTBot IP enumerating the Steam catalog. The model performance number itself is roughly intact (a model is calibrated regardless of who calls it), but the framing implied 11,921 user-driven forecasts when reality was ~119. The above held-out + cohort numbers are the cleaner methodology and what this report now leads with. We're documenting the correction here because hiding it would be the exact kind of non-disclosure this report criticizes.

Why the rest of the industry doesn't publish this

There are three plausible explanations for why commercial Steam-revenue dashboards don't show their coverage numbers, and they're all commercial decisions rather than technical ones.

The first possibility is that they simply don't measure it. Building a held-out backtest pipeline is non-trivial: you need a clean train/test split, an outcomes pipeline that pulls realized revenue from independent sources, and the operational discipline to refresh the comparison whenever the model changes. Most commercial dashboards prioritize coverage of the catalog over disclosure of their own accuracy, because the user-facing "we cover 95% of Steam apps" number sells better than "our 80% cones land 76% of the time on indie debuts." This is not a moral failure — it's a product choice driven by what buyers ask about.

The second possibility is that they do measure it and the number is bad enough that publishing it would erode the price premium. We have no way to know which dashboards fall into this bucket. But it's worth noticing that the silence is asymmetric: a tool whose accuracy is genuinely strong has every commercial incentive to publish, because public coverage numbers are a competitive moat. A tool whose accuracy is weak has every commercial incentive to talk about catalog size and update cadence instead.

The third possibility is structural: rule-of-thumb tools (the Boxleiter family) can't have probabilistic coverage because they don't emit probabilistic outputs. A point estimate has no cone. The honest thing to do is what Boxleiter himself does — disclose the rule's miss-by-30% rate explicitly. Most downstream tools that wrap Boxleiter just bury this.

The cost of an uncalibrated forecast

Consider an indie team finishing a strategy game. They ran a $40K Kickstarter that closed at $52K. The studio's two leads have been working at half-salary for fourteen months. They're trying to decide whether to (a) hire a community manager to help marketing for the last sprint to launch, (b) commit to a Switch port budget, or (c) run a paid wishlist campaign in the final eight weeks.

They plug their wishlist count into a Boxleiter calculator: 28,000 wishlists × $25 × multiplier 50 = $35M lifetime revenue. They know enough to discount that aggressively. They reach for a "more sophisticated" commercial dashboard that estimates first-month revenue at $185,000 with a green confidence indicator. They make the hire and commit to the port — total burn ~$45K over launch quarter — on the basis that ~$185K is comfortably enough to cover it.

The launch lands at $42,000 first-month, full-stop. They are now looking at a fall-quarter cash crunch they didn't see coming. The uncalibrated dashboard's "green confidence indicator" wasn't measuring confidence in the prediction — it was rendering a brand color. Nobody reading it knew the tool's actual miss rate, because the tool didn't publish one.

This story is composite, not specific, and we have heard variants of it often enough that the pattern is clear: a 24% miss-by-30%+ rate becomes catastrophic when the user thinks they're working with calibrated probabilities. The mismatch between the rendered confidence (a slick UI, a precise dollar figure, a green icon) and the underlying error rate (which the user has no way to see) is what causes real harm.

What "good" looks like

If you're choosing a Steam revenue forecasting tool — for your own studio, for a publisher pitch deck, or for a research project — here are the questions that separate calibrated tools from confidence theater.

1. Does the tool publish per-genre coverage?

One global accuracy number hides the systematic per-genre bias that always shows up when you stratify. Action games have different revenue distributions than puzzle games. A tool that reports a single 75% coverage number across the whole cohort is averaging across heterogeneous strata and burying the strata where it's actually dangerous. Per-cluster coverage exposes which genres the tool can be trusted on and which it can't.

2. Is the validation held-out?

If a tool is trained on launches through 2025 and validated on launches from 2024, you're reading numbers from a model evaluated on its own training data. Held-out means: the validation cohort was specifically excluded from the fit, and the prediction is generated as if the launch hadn't happened yet. Demanding this question gets answered in writing is the highest-leverage thing a buyer can do. Tools that resist the question almost always have a reason.

3. Does the tool publish its wrong predictions?

Good tools publish where they were wrong, by name, with a forensic attached. This is a cultural marker more than a technical one — it signals that the maintainers are willing to have a conversation about their failure modes. We publish ours on /methodology. If a vendor refuses to do this even in aggregate, treat their predictions as marketing.

4. Does the tool re-calibrate as the cohort shifts?

The Steam launch distribution in 2026 is not the same as in 2022. Refund-policy changes, the Popular Upcoming algorithm tweaks, the rise of demo-driven discovery, the maturation of the indie F2P sub-market — all of this drifts the underlying distribution. A model trained two years ago and never recalibrated is going to silently degrade. Online recalibration techniques (the technical name is DtACI, drift-tolerant adaptive conformal inference) handle this without retraining the whole model from scratch. Ask whether the tool you're using does this, or whether it's frozen at training time.

Methodology, lightly

For readers who want the technical lineage rather than just the headlines: our forecast cones are produced by Mondrian-asymmetric conformal quantile regression, which is a long phrase for "split the calibration step per genre cluster, and let the upper and lower cone widths adapt independently rather than assuming symmetric error." Conformal prediction (CQR specifically) is a well-studied family of techniques in the modern statistical literature; the Mondrian extension lets per-genre miscoverage be measured and corrected separately. The output is a cone with calibrated 80% coverage per stratum, not just on average.

On top of that we run DtACI online recalibration as new outcomes land. As 2026-cohort launches drift away from the 2024-cohort training distribution, the calibration step shifts to track the new reality without a full model retrain. This is what lets the live-coverage table refresh daily and stay honest about distribution shift.

Both of these techniques are public-domain methodological literature. Anyone can replicate them. The thing that's hard to replicate is the underlying labeled corpus — pulling realized revenue out of Steam's API takes a sustained outcomes pipeline, and that's where most "calibration" projects bog down. The full technical write-up, including the 5-block JSON-LD methodology disclosure for AI-crawler ingestion, is on /methodology.

If you take one thing from this report

It's this: every Steam revenue forecast you'll see this year is doing something measurable to the underlying data. The ones worth your money will tell you what they did and what fraction of the time they were wrong. The ones not worth your money will render a slick UI and decline the question. Pick the first kind.

If you want to read the live calibration evidence yourself, our /methodology page is updated daily and shows the same table our ops dashboard reads. If you want a free single-game forecast for your own project, the homepage tool runs on the same calibrated cones described in this report — drop in your appid and get back a P10–P90 cone with a per-genre coverage badge attached. If you'd rather a deeper, authored launch report with counterfactual marketing-lever scenarios, the /pricing page covers the $299 single-launch report tier.

Sources & companion material

This is the first in a quarterly series. The next report (Q3 2026) will dig into wishlist-conversion-rate distribution shifts post-2026 Refund Policy update. To get notified, drop your email on the homepage and we'll send a single-link email when each new report ships.