Holdout Labs · On-chain Indicator Audit · Batch 1 · 2026-09-25
Twelve famous Bitcoin cycle indicators, tested only after they were published
Each rule frozen exactly as its creator published it, tested only on data after its first publication, with the protocol sealed by RFC 3161 timestamps (DigiCert, FreeTSA) before any on-chain data were fetched. Every result is published.
Family A: price and block-header indicators
Result: One claim had enough independent episodes to test and was not supported; five produced too few episodes for a reliable conclusion. Stock-to-flow, the only testable one, went the wrong way. The other five fired 1–4 times after publication, too few for any honest inference, so we report them descriptively; all five pointed the way their creators claimed, which is what a handful of cycles can do by chance. All four implied trading rules graded F against buy-and-hold BTC: none showed a statistically robust edge over holding.
| Indicator | Frozen rule (primary zone) | Episodes | Avg 365-day return after zone days vs all days | Status (Holm, m=6) | Implied-rule audit |
|---|
Pi Cycle Top
Philip Swift, Apr 2019 | crossing day
top claim | 1 | -33% vs +42% | NOT TESTABLE
descriptive only | no implied rule |
200-week MA heatmap (bottom claim)
PlanB concept, Jan 2019 | close <= 1.10 x 200WMA
bottom claim | 3 | +102% vs +42% | NOT TESTABLE
descriptive only | no implied rule |
Mayer Multiple
Trace Mayer, 2017 | MM > 2.4
top claim | 3 | -21% vs +36% | NOT TESTABLE
descriptive only | F excess +3%/yr |
2-Year MA Multiplier
Philip Swift, Jul 2017 | close < 2yMA
bottom claim | 4 | +108% vs +33% | NOT TESTABLE
descriptive only | F excess -13%/yr |
Puell Multiple
David Puell, Mar 2019 | Puell < 0.5
bottom claim | 4 | +105% vs +42% | NOT TESTABLE
descriptive only | F excess +0%/yr |
Stock-to-flow (PlanB 2019)
22 Mar 2019 | close < S2F model
bottom claim | 6 | +26% vs +42% | NOT SUPPORTED
p 0.94, Holm 1.00 | F excess -19%/yr |
Out-of-sample windows start the first day after publication (or when the indicator can be computed) and end 2026-09-24. Returns are geometric means of 365-day returns over days with a full window. Fewer than 5 episodes = descriptive only (preregistered).
What we found
- Stock-to-flow. The only primary claim with enough episodes to test (6). Days below the model price were followed by lower 365-day returns than average, the opposite of the claim (one-sided p = 0.94; Holm-adjusted 1.00): not supported. As a level model it drifted away: BTC closed at 0.15x the model on average in 2025 and 0.09x in 2026 (model price on 2026-09-24: $862,472).
- Pi Cycle Top. One out-of-sample crossing, on 2021-04-12. It came one day before the April 2021 local high (context, not a preregistered test), but the highest close within a year either side came 210 days later, on 2021-11-08, and 12.9% higher, so it misses the preregistered 'within 3 days' test at every window. No crossing at all before the highest closes of 2024-03 or 2025-10.
- Mayer Multiple > 2.4. Three episodes (Jan 2018, Jun 2019, Jan–Mar 2021). The first came 20 days after the December 2017 high and the second on the day of the 2019 high; the 2021 one came ten months before the cycle high. None since March 2021. Its implied rule's average excess return over buy-and-hold was +3.1 points a year (compound 27.5% vs 23.0%), all of it from those early episodes: not significant (SPA p = 0.12), and identical to holding throughout the holdout, so it grades F.
- 2-Year MA Multiplier. Four bottom-zone episodes with a full 365-day window (plus one open in 2026); the 5x top zone fired only in late 2017. Too few for inference.
- Puell Multiple. Never above 4 after publication; four episodes below 0.5. The implied rule was long for the whole window, so it was buy-and-hold.
- 200-week MA. Three bottom-zone episodes with a full 365-day window (plus two open in 2026). Price spent 13% of out-of-sample days below the 200-week MA.
Family B: UTXO metrics, computed by us from the public blockchain
MVRV Z-score, NUPL, SOPR, Reserve Risk, short-term-holder cost basis and realized price, sealed in the same preregistration, computed by us from the public Bitcoin blockchain (Google BigQuery public dataset), so no vendor licence is involved. The data-source change and the creator-source checks were sealed before any value was computed. Our series match published Coin Metrics and Glassnode figures within a 0.2% median difference.
Result: two claims were testable (SOPR reset, short-term-holder cost basis) and neither was supported; four fired 1–3 times after publication and are descriptive only. The STH cost-basis rule graded F: no statistically robust edge over holding on average excess return, although it compounded at 28.5% a year vs 18.3% with a 33% vs 77% maximum drawdown (different measures, pointing different ways).
| Metric | Frozen rule (primary zone) | Episodes | Avg return after zone days vs all days | Status (Holm, m=6) | Implied-rule audit |
|---|
MVRV Z-score
@aweandwonder, 9 Oct 2018 | MVRV-Z > 5 | 1 | -15% vs +44%, 365d | NOT TESTABLE
descriptive only | no implied rule |
NUPL
Adamant Research, Feb 2019 | NUPL < 0 | 3 | +82% vs +42%, 365d | NOT TESTABLE
descriptive only | no implied rule |
SOPR (unadjusted)
Renato Shirakashi, 25 Apr 2019 | close > 200-day SMA and SOPR 7-day SMA <= 1 | 13 | +17% vs +7%, 90d | NOT SUPPORTED
p 0.20, Holm 1.00 | no implied rule |
Reserve Risk
Hans Hauge, 30 May 2019 | Reserve Risk < 0.0026 | 2 | +84% vs +42%, 365d | NOT TESTABLE
descriptive only | no implied rule |
Short-term-holder cost basis
Glassnode, 2020 | close > STH cost basis | 7 | +4% vs +3%, 90d | NOT SUPPORTED
p 0.26, Holm 1.00 | F avg excess +0.5 pts/yr; compound 28.5% vs 18.3% |
Realized price
Carter & Le Calvez, Oct 2018 | close < realized price | 3 | +94% vs +45%, 365d | NOT TESTABLE
descriptive only | no implied rule |
- MVRV Z-score > 5. One episode after publication, 2 Jan to 15 Apr 2021 (peak 6.9 on 21 Feb 2021). The highest close within a year either side came on 2021-11-08, 310 days after the zone began, so it misses the creator's 'within two weeks' claim at every window (14, 30, 90 days). In the 2024-2026 cycle Z peaked at 3.3 (Dec 2024) and never reached 5. The 365-day return after zone days averaged -15% vs +44% on all days, from that single episode: descriptive only.
- NUPL < 0. Three episodes (the window's first weeks in Mar 2019, Mar 2020, Jun 2022 to Jan 2023), followed by an average 365-day return of +82% vs +42% on all days. Three episodes are too few for inference. The 'euphoria' band (>= 0.75) was never reached after publication (peak 0.747 in 2021).
- SOPR reset in a bull regime. Testable: 13 episodes. The 90-day return after reset days averaged +17% vs +7% on all days, but the circular-shift null produces gaps this large often (one-sided p = 0.20; Holm-adjusted 1.00): not supported. The creator's 10-day smoothing gives the same picture (exploratory).
- Reserve Risk < 0.0026. Below the band on 75% of out-of-sample days, and continuously since 21 Jan 2022: the HODL Bank denominator keeps growing, so a fixed band drifts into being 'on' almost all the time (every day of 2023 and 2026). That leaves two episodes: descriptive only. It never exceeded the 0.02 top band after publication (2021 peak 0.0094).
- Short-term-holder cost basis. Testable: 7 bull-regime episodes, about half of all days. The 90-day return after days above the STH cost basis averaged +4.0% vs +2.6% on all days (p = 0.26; Holm-adjusted 1.00): not supported. Its implied rule (hold above, cash below) grades F against buy-and-hold: average excess return +0.5 points a year (excess Sharpe +0.01, SPA p = 0.52), no statistically robust edge. On compound growth it did much better, 28.5% a year vs 18.3% for holding, with a maximum drawdown of 33% vs 77%. These are different measures pointing different ways; the sealed objective (a) tests the average, and drawdown reduction (objective (b)) was not declared, so no grade is given for it.
- Realized price. Price closed below the realized price in three episodes (Nov 2018 to Mar 2019, Mar 2020, Jun 2022 to Jan 2023), followed by an average 365-day return of +94% vs +45% on all days. These are the same days as MVRV-Z < 0 (the two zones are identical by construction). Three episodes: descriptive only.
Family C: continuous-level and pooled tests (preregistered, not blind)
Not blind: written after we had seen the price path and the Family A/B results; sealed before any Family C statistic was computed. Result: under the sealed test 4 of 24 tests reached significance, but a calibration check (exploratory) shows that test is too lenient for metrics this persistent; calibrated, none is significant (smallest p 0.07 before correction). No statistically robust association between any metric's level and later BTC returns. Descriptively the valuation metrics pointed the claimed way, over about two market cycles. Association is not a tradable edge; no rule is graded.
Limits
- Not blind: we had seen BTC prices through 2026; four indicators are pure functions of price. Protection comes from rules taken from the creators, sealed before the run, and full publication.
- Tiny samples: cycle-top signals fired 0–3 times after publication. One or two events cannot confirm or refute a claim.
- Our operationalisations (sealed in advance): “close to the 200-week MA” = within 10%; 5x the 2-year MA = reached; daily stock-to-flow; Glassnode's Puell bands; Mayer window from 2018-01-01.
Seal
Preregistration SHA-256 aafe3c37f5bbf83bc6e4b9a986ef1c936c3b74a927c253b833a5a7626482de40
Seal record SHA-256 f8d7d587466a4bb61b5c179aba44bb766825a6a495cc3ecb68d16e39c1810275
Signed: digicert 2026-09-25T18:22:38Z, freetsa 2026-09-25T18:22:38Z
Block headers fetched over P2P at 2026-09-25T18:23:09Z (968,471 headers, PoW-checked).
Prices from the Coinbase Exchange public market-data API; derived statistics only, never raw prices; no redistribution. Statistical analysis of past data. Not investment advice; no recommendation to buy, sell or hold anything.