On-chain Indicator Audit · Batch 1
Twelve famous Bitcoin cycle indicators, tested only after they were published
Every indicator has a birthday: the day its creator published it. The days after it are a free out-of-sample test of a rule nobody can re-tune. We froze twelve rules exactly as first published and sealed the whole test before fetching any on-chain data (we had already seen Bitcoin’s price path; disclosed below). Three claims had enough independent episodes to test, and none was supported; nine can’t be validated yet: only 1 to 4 signals each since publication. All five implied trading rules graded F.
A follow-up, Family C (preregistered, not blind), tested every day’s metric level. Its sealed test flagged 4 of 24 tests as associated with later returns; we then found that test’s null too lenient for such persistent series, and after a calibration check none is significant. We do not treat the four as robust evidence.
Data vendors sell these metrics. We test whether they carry an out-of-sample edge. MVRV Z-score, NUPL, SOPR, Reserve Risk, short-term-holder cost basis and realized price were computed by us from the raw blockchain.
How it was done
- Frozen as first published. Each rule, threshold and moving-average length is taken from the creator’s publication, with its date. The test window starts the day after.
- Preregistered, then sealed. One document fixed the rules, windows, forward-return horizons (90/180/365 days), the null (a circular-shift test that keeps each signal’s timing structure), Holm’s correction within each family of six, a rule that fewer than 5 independent episodes means “descriptive only”, and a promise to publish everything. Timestamped by DigiCert and FreeTSA (RFC 3161) on 2026-09-25 at 18:22:38 UTC.
- Our own chain data. Puell Multiple and stock-to-flow need daily issuance. We fetched all 968,471 block headers straight from the Bitcoin peer-to-peer network after the seal, and checked every one’s link and proof of work. The six UTXO metrics (Family B) we computed ourselves from the public Bitcoin blockchain via Google BigQuery, and cross-checked against published values.
- Run once. Indicators that imply a trading rule were also run through our full audit (rubric v1.1) against simply holding Bitcoin.
What we found: Family A
Six price and block-header indicators.
Stock-to-flow was the only claim with enough episodes (6) to test. Days below the model price were followed by lower 365-day returns than average, the opposite of the claim (p = 0.94). As a level model it drifted away: Bitcoin averaged 0.15× the model price in 2025 and 0.09× in 2026.
Cycle-top signals barely fire. Pi Cycle crossed once after publication (2021-04-12, a day before the April 2021 local high, but 210 days before the cycle’s highest close); it did not cross before the 2024 or 2025 highs. The Puell Multiple never went above 4. Descriptively, all five small-sample indicators pointed the way their creators said, but one to four episodes is what a handful of cycles looks like, not evidence.
Implied rules: all four graded F against holding Bitcoin, net of costs. The Mayer Multiple rule was ahead on both average excess return (+3.1 points a year) and compound growth (27.5% vs 23.0%), but not significantly, and it matched holding throughout the holdout.
Statistics on past data only. Not a verdict on on-chain analysis in general, and not a recommendation to use or avoid any indicator.
Before vs after publication
How did each rule look on the history its creator could see when designing it, and how did it look after it was published? This is a description, not a test: no p-values, no grades. The sealed results on this page are unchanged.
Top signals (Pi Cycle, Mayer, MVRV-Z) flagged 8 of 9 cycle tops before publication and 2 of 9 after.
Both later hits were the April 2021 high. None came near the Nov 2021 or Oct 2025 highs.
Bottom signals did not follow this pattern. Several (2-year MA, Puell, NUPL, realized price) looked no better than an average day before publication, and better after. Stock-to-flow and Reserve Risk “caught” lows after publication mainly because they were on most of the time (79% and 75% of days).
Read with care
- Descriptive, not a test. Each window holds only 1 to 4 cycles. Rules tend to look good on the history they were designed on, and chance alone can make “before” look better than “after”.
- What counts as a cycle. A cycle top is an all-time high followed by a fall of at least 50% before the next high; the cycle low is the lowest close in between. A stricter version (a fall of at least 75%) leaves fewer tops and tells the same story: the three top signals called 2 of 2 before and 0 of 1 after.
- Early data are thin. Our prices start on 2011-08-18, from Bitstamp, then a small, thinly traded exchange. Earlier peaks (2010, June 2011) are not covered.
- Early on-chain values are distorted. Coins created before 2011-08-18 count as costing nothing, which drags early realized price down (and pushes NUPL and MVRV up), most in 2011 and 2012.
- Secondary top zones. The top bands of Puell, NUPL and Reserve Risk flagged every pre-publication top, then never fired after publication.
Full tables, method and cycle dates: BEFORE-AFTER.md (a trimmed copy: three price figures in the cycle definition removed; nothing else changed). In the tables below, the grey line under each indicator gives its before → after summary, and its Why? has the detail.
Family A: price and block-header indicators
| Indicator | First published | Frozen rule tested | Window from | Episodes | Avg 365-day return after signal vs all days | p / Holm p | Claim | Implied-rule grade (average excess return vs holding) | Why? |
|---|---|---|---|---|---|---|---|---|---|
| Pi Cycle TopTops called: 2 of 3 before → 1 of 3 after | Philip Swift, Apr 2019 | crossing day (top) | 2019-05-01 | 1 | −33% vs +42% | – | Can’t be validated yet: only 1 signal since publication | Not graded: no re-entry rule published† | Why? Pi Cycle TopPi Cycle Top The claim. When the 111-day average price rises above twice the 350-day average, a cycle high is within 3 days. What happened. After publication it crossed once, on 12 Apr 2021, a day before that April’s local high. But the cycle’s highest close came 210 days later and 13% higher, and there was no crossing before the 2024 or 2025 highs. Why no verdict. One signal can’t tell skill from luck, so we only describe it. And no grade: the creator published no rule for when to get back in or out, so there is no complete rule to test against holding. Before vs after publication. Before publication it fired within 3 days of 2 of the 3 cycle tops in our data (4 days early on the other). After, 1 of 3: April 2021, with no signal near the Nov 2021 or Oct 2025 highs. For interest (not a test). Taking the published sell signal at face value: holding from 1 May 2019, selling at the one crossing (12 Apr 2021) and staying in cash to 24 Sep 2026 grew $1 to $11.25 (38.7% a year), against $15.86 (45.3% a year) for simply holding. Its worst fall was −62% (March 2020, before the sale), against −77%. We did not work out “sell, then buy back X days later”: no buy-back rule was published, and picking X would be inventing one. This rests on one decision, and we chose the end date, so it isn’t evidence either way. |
| 200-week MA heatmapLows caught: 1 of 1 before → 2 of 3 after | PlanB concept, Jan 2019 | close <= 1.10 x 200WMA (bottom) | 2019-05-19 | 3 | +102% vs +42% | – | Can’t be validated yet: only 3 signals since publication | Not graded: no exit rule published† | Why? 200-week MA heatmap200-week MA heatmap The claim. Times when price is close to its 200-week average (about four years of weekly prices) have been followed by strong gains. We read “close” as within 10%. What happened. This happened on 3 separate occasions with a full year of data after (Mar 2020, Jun 2022 to Jun 2023, Jul to Oct 2023), plus two in 2026 too recent to measure. The year after averaged +102%, against +42% after a typical day. Why no verdict. We set in advance that fewer than 5 separate occasions is too few to tell skill from luck, so this is a description, not evidence. And no grade: the creator published no rule for when to get back in or out, so there is no complete rule to test against holding. Before vs after publication. In its pre-publication data, price came near or below the line within 30 days of the one cycle low (Dec 2018). After, 2 of 3 lows (Nov 2022 and the 2026 low so far; not July 2021). For interest (not a test). Bitcoin long-term holders rarely sell, so here is what the published buying zone would have meant. Waiting for the zone meant a single purchase, on 12 Mar 2020; holding that to 24 Sep 2026 grew $1 to $17.37 (47.5% a year), against $10.29 (37.3% a year) for buying when our window opens (19 May 2019), with the same −77% worst fall. This rests on one decision, and we chose the end date, so it isn’t evidence either way. |
| Mayer MultipleTops called: 3 of 3 before → 0 of 3 after | Trace Mayer, 2017 | MM > 2.4 (top) | 2018-01-01 | 3 | −21% vs +36% | – | Can’t be validated yet: only 3 signals since publication | F (+3%/yr) | Why? Mayer MultipleMayer Multiple The claim. When price is more than 2.4 times its 200-day average, the market is overheated and weaker returns follow. What happened. Three occasions (Jan 2018, Jun 2019, Jan to Mar 2021), none since. The year after averaged −21%, against +36% after a typical day. Three is too few to be evidence. Why F. The graded rule holds Bitcoin except when the multiple is above 2.4, then sits in cash. It finished ahead of simply holding (27.5% vs 23.0% a year), but the lead came from about four weeks out of the market in 2018 to 2021, and a lead that size shows up by chance about 1 time in 8. In the final 30% of the period, kept aside as a check, it did exactly what holding did. Before vs after publication. Before publication it was above 2.4 within 30 days of all 3 cycle tops. After, 0 of 3: its last signal ended 31 days before the April 2021 high, and there was nothing near the Nov 2021 or Oct 2025 highs. |
| 2-Year MA MultiplierLows caught: 1 of 1 before → 3 of 4 after | Philip Swift, Jul 2017 | close < 2yMA (bottom) | 2017-08-01 | 4 | +108% vs +33% | – | Can’t be validated yet: only 4 signals since publication | F (−13%/yr) | Why? 2-Year MA Multiplier2-Year MA Multiplier The claim. Price below its 2-year average has marked cycle bottoms; price near 5 times that average has marked tops. What happened. Price fell below the 2-year average on 4 occasions with a full year after (Nov 2018, Oct 2019, Mar 2020, May 2022). The year after averaged +108%, against +33% after a typical day, but four is too few to be evidence. The 5× level was reached only in late 2017. Why F. The +108% counts only the days the signal was on. The grade tests the whole rule: start in cash, hold Bitcoin once price drops below the 2-year average, go back to cash at 5 times it. It started in cash in Aug 2017 and missed that year’s rise, first held Bitcoin in Nov 2018, and never left, because 5× never came again. Result: 34.7% a year against 44.7% for holding; in the final 30% of the period it matched holding exactly. Before vs after publication. Before publication, the year after days below the line was no better than after any day (+104% vs +105%), though it did catch the one cycle low in its data (Jan 2015). After, it caught 3 of 4 lows (not July 2021), and the year after averaged +108% vs +33%. |
| Puell MultipleLows caught: 2 of 3 before → 2 of 3 after | David Puell, Mar 2019 | Puell < 0.5 (bottom) | 2019-04-01 | 4 | +105% vs +42% | – | Can’t be validated yet: only 4 signals since publication | F (+0%/yr) | Why? Puell MultiplePuell Multiple The claim. The Puell Multiple compares the dollar value of newly mined bitcoin each day with its one-year average. Below 0.5 has marked bottoms; above 4, tops. What happened. Below 0.5 on 4 occasions (Mar 2020, May to Jul 2020, Jun 2021, Jun to Dec 2022). The year after averaged +105%, against +42% after a typical day, but four is too few to be evidence. It never went above 4. Why F. The graded rule holds Bitcoin after a reading below 0.5 and moves to cash after a reading above 4. It was already holding when the test window began and, with no reading above 4, never moved to cash. So it simply was holding: 49.8% a year, exactly the same. The test asks for a reliable edge over holding, and matching it is not one. Before vs after publication. It was below 0.5 near 2 of 3 cycle lows both before and after publication. Before publication, the year after its signal days was weaker than after a typical day (+84% vs +130%); after, stronger (+105% vs +42%). |
| Stock-to-flowLows caught: 3 of 4 before → 3 of 3 after (on 79% of days after) | PlanB, 22 Mar 2019 | close < S2F model (bottom) | 2019-03-23 | 6 | +26% vs +42% | 0.94 / 1.00 | Not supported | F (−19%/yr) | Why? Stock-to-flowStock-to-flow The claim. Scarcity sets value: the coins in existence divided by the new coins mined each year give a “model price”, and price should move toward it. So days with price below the model should be followed by gains. What happened. 6 occasions, enough to test. Days below the model were followed by weaker year-ahead returns (+26%) than a typical day (+42%), the opposite of the claim. The model also drifted away: in 2026 price averaged 0.09 times the model price. Why F. The graded rule holds Bitcoin while price is below the model and sits in cash above it. It was in cash for much of 2019 to 2021 while price rose, and trailed holding in each of those years. Since April 2021 price has stayed below the model, so the rule has been holding ever since. Overall: 32.8% a year against 50.1% for holding. Before vs after publication. Price was below the model near 3 of 4 cycle lows before publication and all 3 after, but only because it has been below the model on 79% of days since. In the data the model was fitted on, below-model days were followed by +428% a year later vs +163% for any day; after, +26% vs +42%. |
Return after signal describes only the days the signal was on. The grade audits the full buy/sell rule against simply holding over the whole window, so the two can disagree. The grey line under each name compares before and after publication (descriptive only). Open Why? on any row for a plain-English explanation.
† Not graded. The creator published an entry (or exit) signal but not a complete in/out strategy, so there is nothing to grade without inventing a rule ourselves.
Windows end 2026-09-24. Returns are geometric means over days with a full 365-day window. “Episodes” are separate runs of signal days (gaps under 30 days merged). With fewer than 5, the preregistered rule reports the result descriptively and shows no p-value. Top claims predict lower returns after the signal; bottom claims predict higher.
Family B: six UTXO metrics, computed by us from the blockchain
MVRV Z-score, NUPL, SOPR, Reserve Risk, short-term-holder (STH) cost basis and realized price were sealed in the same preregistration, but no vendor licence we checked allowed commercial publication. So we built them ourselves.
Method. Computed ourselves from the public Bitcoin blockchain via Google BigQuery (every output’s creation and spending day), valued locally, and cross-checked against published values. No vendor data was used. Our supply matched the consensus issuance schedule within 80 BTC. Realized cap, realized price and MVRV matched 25 published Coin Metrics and Glassnode figures with a median difference of 0.2%; STH cost basis matched 9 published figures with a median difference of 0.35%. One Glassnode prose figure (an MVRV of 1.32, January 2025) is 83% off ours and inconsistent with Glassnode’s own realized-cap figures; we show it unedited and excluded it from the tolerance check.
Sealed before any metric was computed. The change of data source, and the checks of each creator’s primary source, were sealed first in a deviation note. That is why the MVRV Z-score top band is the creator’s 5, not the commonly quoted 7 we originally sealed: the sealed text said the creator’s primary source wins. Family B is its own Holm family of six.
What we found: Family B
Two claims were testable, and neither was supported. The SOPR reset in a bull regime had 13 episodes: the 90-day return after reset days averaged +17% against +7% on all days, but the circular-shift null produces gaps this large often (p = 0.20, Holm 1.00). Days above the STH cost basis (7 episodes) were followed by a 90-day return of +4.0% against +2.6% on all days (p = 0.26, Holm 1.00).
Cycle tops barely fire. MVRV Z-score went above 5 once after publication (January to April 2021); the highest close came 310 days after the zone began, missing the “within two weeks” claim. In the 2024–2026 cycle it peaked at 3.3. NUPL never reached its 0.75 “euphoria” band (peak 0.747).
Bottom zones. NUPL below 0, price below realized price and low Reserve Risk each fired 2 or 3 times. Descriptively, all pointed the way their creators said, but that is what a handful of cycles looks like, not evidence. Reserve Risk sat in its bottom band on 75% of days after publication, and continuously since January 2022: a fixed band drifts into being on almost all the time.
Statistics on past data only. Not a verdict on on-chain analysis in general, and not a recommendation to use or avoid any indicator.
Family B: UTXO metrics
| Metric | First published | Frozen rule tested | Window from | Episodes | Avg return after signal vs all days (horizon) | p / Holm p | Claim | Implied-rule grade (average excess return vs holding) | Why? |
|---|---|---|---|---|---|---|---|---|---|
| MVRV Z-scoreTops called: 3 of 3 before → 1 of 3 after | @aweandwonder, 9 Oct 2018 | MVRV-Z > 5 (top) | 2018-10-10 | 1 | −15% vs +44% (365d) | – | Can’t be validated yet: only 1 signal since publication | Not graded: no complete in/out rule published† | Why? MVRV Z-scoreMVRV Z-score The claim. MVRV Z compares Bitcoin’s market value with its “realized” value (each coin priced when it last moved, roughly what holders paid), scaled by how much market value usually swings. Above 5 marks a cycle top within about two weeks. What happened. Above 5 once, from Jan to Apr 2021. The cycle’s highest close came 310 days after that began. In the 2024–2026 cycle it peaked at 3.3. Why no verdict. One occasion can’t tell skill from luck, so we only describe it. And no grade: the creator published no rule for when to get back in or out, so there is no complete rule to test against holding. Before vs after publication. Before publication it was above 5 within 14 days of all 3 cycle tops. After, 1 of 3 (April 2021); it never reached 5 near the Nov 2021 or Oct 2025 highs. |
| NUPLLows caught: 2 of 4 before → 1 of 3 after | Adamant Research, Feb 2019 | NUPL < 0 (bottom) | 2019-03-01 | 3 | +82% vs +42% (365d) | – | Can’t be validated yet: only 3 signals since publication | Not graded: no complete in/out rule published† | Why? NUPLNUPL The claim. NUPL is the share of Bitcoin’s market value that is paper profit (market value minus what holders paid, as a share of market value). Below 0, holders on average are at a loss, which has marked bottoms; 0.75 and above (“euphoria”) has marked tops. What happened. Below 0 on 3 occasions (Mar 2019, Mar 2020, Jun 2022 to Jan 2023). The year after averaged +82%, against +42% after a typical day. It never reached 0.75 (peak 0.747 in 2021). Why no verdict. Three occasions are too few to tell skill from luck. And no grade: the creator published no rule for when to get back in or out, so there is no complete rule to test against holding. Before vs after publication. Before publication it was below 0 near 2 of 4 cycle lows (Jan 2015 and Dec 2018; not 2011 or mid-2013, where our early values are least reliable). After, 1 of 3 (Nov 2022 only). |
| SOPR (unadjusted)90-day return, signal vs any day: +29% vs +29% before → +17% vs +7% after | Renato Shirakashi, 25 Apr 2019 | close > 200-day SMA and SOPR 7-day SMA <= 1 (bull-regime reset) | 2019-04-26 | 13 | +17% vs +7% (90d) | 0.20 / 1.00 | Not supported | Not graded: no exit rule published† | Why? SOPRSOPR The claim. SOPR compares the price coins are spent at with the price they last moved at (above 1 means spent at a profit). In a rising market (price above its 200-day average), dips to 1 or below have been followed by gains. What happened. 13 occasions, enough to test. The 90 days after averaged +17%, against +7% after a typical day. Why not supported. When we slid the signal days to random other positions in the same price history, gaps that large turned up about 1 time in 5, so this could easily be chance. There is no grade: no complete in-and-out rule was published to test. Before vs after publication. Before publication, the 90 days after reset days were no better than after any day (+29% vs +29%). After, +17% vs +7%, a gap the sealed test found could easily be chance. |
| Reserve RiskLows caught: 2 of 4 before → 2 of 3 after (on 75% of days after) | Hans Hauge, 30 May 2019 | Reserve Risk < 0.0026 (bottom) | 2019-05-31 | 2 | +84% vs +42% (365d) | – | Can’t be validated yet: only 2 signals since publication | Not graded: no complete in/out rule published† | Why? Reserve RiskReserve Risk The claim. Reserve Risk compares price with long-term holders’ conviction (roughly, how much they have given up by not selling). Below 0.0026 has marked bottoms, followed by strong gains. What happened. It was below 0.0026 on 75% of days after publication, and on every day since Jan 2022: the conviction measure keeps growing, so a fixed line ends up switched on almost all the time. That leaves only 2 separate occasions; the year after averaged +84%, against +42% after a typical day. Why no verdict. Two occasions are too few to tell skill from luck. And no grade: the creator published no rule for when to get back in or out, so there is no complete rule to test against holding. Before vs after publication. It was below the band near 2 of 4 cycle lows before publication and 2 of 3 after, but since publication it has been below the band on 75% of days (every day since Jan 2022). |
| Short-term-holder cost basis90-day return, signal vs any day: +40% vs +26% before → +4% vs +3% after | Glassnode, 2020 | close > STH cost basis (bull regime) | 2021-07-01 | 7 | +4% vs +3% (90d) | 0.26 / 1.00 | Not supported | F (+0.5 pts/yr) | Why? Short-term-holder cost basisShort-term-holder cost basis The claim. The short-term-holder cost basis is the average price paid for coins that moved in roughly the last 155 days. Price above it marks a rising market, with better returns to follow. What happened. 7 occasions, enough to test. The 90 days after averaged +4.0%, against +2.6% after a typical day, but random shifts of the signal days produced gaps that large about 1 time in 4, so the claim is not supported. Why F. The graded rule holds Bitcoin above the line and sits in cash below it. The sealed test asked whether it earned more than holding on an average day: +0.5 points a year, too small to tell from zero. It did grow faster overall (28.5% vs 18.3% a year), with a worst fall of 33% against 77%, mostly by sitting out steep declines. Smaller losses were not the question set in advance, so they earn no grade. Before vs after publication. Before publication, the 90 days after days above it averaged +40%, against +26% after any day. After publication, +4% vs +3%. |
| Realized priceLows caught: 1 of 3 before → 2 of 4 after | Carter & Le Calvez, Oct 2018 | close < realized price (bottom) | 2018-11-01 | 3 | +94% vs +45% (365d) | – | Can’t be validated yet: only 3 signals since publication | Not graded: no exit rule published† | Why? Realized priceRealized price The claim. Realized price is the average price at which every bitcoin last moved, roughly what holders paid on average. Price below it has marked cycle bottoms. What happened. Below it on 3 occasions (Nov 2018 to Mar 2019, Mar 2020, Jun 2022 to Jan 2023). The year after averaged +94%, against +45% after a typical day. These are the same days as MVRV Z-score below 0. Why no verdict. Three occasions are too few to tell skill from luck. And no grade: the creator published no rule for when to get back in or out, so there is no complete rule to test against holding. Before vs after publication. Before publication, price was below it near 1 of 3 cycle lows (Jan 2015; not 2011 or mid-2013, where our early values are least reliable). After, 2 of 4 (Dec 2018 and Nov 2022; not July 2021 or the 2026 low so far). |
Return after signal describes only the days the signal was on. The grade audits the full buy/sell rule against simply holding over the whole window, so the two can disagree. Only one rule here (STH cost basis) could be graded; the others have no published rule for getting in and out. The grey line under each name compares before and after publication (descriptive only). Open Why? on any row for a plain-English explanation.
† Not graded. The creator published an entry (or exit) signal but not a complete in/out strategy, so there is nothing to grade without inventing a rule ourselves.
Windows end 2026-09-24. Each metric’s primary horizon was sealed in advance: 365 days for the top and bottom zones, 90 days for the two regime rules. Episodes, the testability rule and the null are as for Family A. Price below realized price covers the same days as MVRV Z-score below 0 (identical by construction).
The one implied rule: STH cost basis (hold Bitcoin above it, cash below). Graded F under the sealed objective, “beat buy-and-hold Bitcoin”, which tests the average excess return: +0.5 points a year (excess Sharpe +0.01, SPA p = 0.52), no statistically robust edge. On compound growth it did much better: 28.5% a year against 18.3% for holding, with a maximum drawdown of 33% against 77%, over 2021-07-01 to 2026-09-24. These are different measures pointing different ways.
Drawdown reduction was not the sealed test, so no grade is given for it. We do not re-grade a rule after seeing its result; testing that claim would need its own preregistration.
Family C: every day’s level, and pooled signals (preregistered, not blind)
Not blind. Family C was written after we had seen Bitcoin’s price path and every Family A and B result. It was preregistered and sealed (FAMILY-C.md, timestamped by DigiCert and FreeTSA on 2026-09-26 at 05:01:04 UTC) before any Family C statistic was computed, but it is not a blind test and should be weighed accordingly.
Why. Nine of the twelve zone rules above fired only 1 to 4 times since publication, so they can’t be validated yet. Family C uses more of the data. C1 asks whether each metric’s level on a day is rank-correlated (Spearman) with Bitcoin’s return over the next 90 or 365 days: 11 metrics, two horizons. NUPL and price / realized price rank days exactly as MVRV does, so they are not counted twice. C2 pools the onsets of all bottom signals, and separately all top signals, and asks whether the 365 days after an onset differed from an average day. The sealed null shifts each metric (or onset series) against the returns by at least 365 days, 2,000 draws. One Holm family of 24 tests.
What we found: Family C
Under the sealed test, 4 of 24 tests were flagged “association supported” after Holm correction: Reserve Risk at 365 days, price / STH cost basis at 90 days, price / 2-year MA at 365 days and price / 200-week MA at 365 days (each Holm p = 0.012). After the seal, while writing the synthetic unit tests, we found that the sealed circular-shift null is too lenient for highly persistent metrics: on surrogate returns unrelated to the metric by construction, a nominal 5% test rejected 5% to 29% of the time. A calibration check we added afterwards (not preregistered; exploratory) gives a smallest p of 0.07 before Holm correction, so none of the 24 is significant after calibration. We do not treat these as robust evidence; the sealed test’s null was miscalibrated, which we found ourselves and disclose.
Even a robust correlation would not be a tradable edge. Association is not a trading rule, and Family C grades none. Overlapping daily returns over 2017–2025 span only about two market cycles. The pooled tests were not significant even under the sealed test (bottom onsets p = 0.028, Holm 0.56; top onsets p = 0.077, Holm 1.00). Stock-to-flow pointed the opposite way to its claim (rho +0.21 at 365 days).
Statistics on past data only. Not a verdict on on-chain analysis in general, and not a recommendation to use or avoid any indicator.
Family C: all 24 tests, sealed verdict and calibration check
| Metric | Horizon | Claimed sign | Window from | Statistic | Sealed p / Holm p (m = 24) | Sealed verdict | Real false-positive rate of a nominal 5% test (exploratory) | Calibrated p, before Holm (exploratory) | Why? |
|---|---|---|---|---|---|---|---|---|---|
| MVRV (= price / realized price) | 90d | negative | 2018-11-01 | rho −0.17 | 0.33 / 1.00 | Not supported | 6% | 0.36 | |
| MVRV (= price / realized price) | 365d | negative | 2018-11-01 | rho −0.49 | 0.16 / 1.00 | Not supported | 5% | 0.22 | |
| MVRV Z-score | 90d | negative | 2018-10-10 | rho −0.18 | 0.33 / 1.00 | Not supported | 8% | 0.34 | |
| MVRV Z-score | 365d | negative | 2018-10-10 | rho −0.51 | 0.15 / 1.00 | Not supported | 8% | 0.21 | |
| SOPR 7-day SMA | 90d | negative | 2019-04-26 | rho −0.03 | 0.51 / 1.00 | Not supported | 8% | 0.49 | |
| SOPR 7-day SMA | 365d | negative | 2019-04-26 | rho −0.14 | 0.34 / 1.00 | Not supported | 12% | 0.43 | |
| Reserve Risk | 90d | negative | 2019-05-31 | rho −0.26 | 0.12 / 1.00 | Not supported | 9% | 0.16 | |
| Reserve Risk | 365d | negative | 2019-05-31 | rho −0.61 | 0.0005 / 0.012 | Association supported | 14% | 0.13 | Why? Reserve Risk, 365 daysReserve Risk, 365 days The question. Across every day, did lower Reserve Risk tend to come before bigger gains over the next year? Yes, fairly strongly (a rank correlation of −0.61, where −1 would be a perfect reverse ordering). Why not counted. The sealed test flagged it. We then found that test raises false alarms too often for metrics that change slowly: on made-up returns unrelated to the metric, it fired 14% of the time instead of 5%. Allowing for that, a result this strong arises by chance about 13% of the time, so it is not significant. Even so. This covers only about two market cycles, and a link between a level and later returns is not a trading rule; nothing here is graded. |
| close / STH cost basis | 90d | positive | 2021-07-01 | rho +0.10 | 0.0005 / 0.012 | Association supported | 15% | 0.10 | Why? Price / STH cost basis, 90 daysPrice / STH cost basis, 90 days The question. Across every day, did price further above the short-term-holder cost basis come before better 90-day returns? Only weakly (a rank correlation of +0.10, where +1 would be a perfect ordering). Over one year, even the sealed test found nothing. Why not counted. The sealed test flagged it. We then found that test raises false alarms too often for metrics that change slowly: on made-up returns unrelated to the metric, it fired 15% of the time instead of 5%. Allowing for that, a result this strong arises by chance about 10% of the time, so it is not significant. Even so. This covers only about two market cycles, and a link between a level and later returns is not a trading rule; nothing here is graded. |
| close / STH cost basis | 365d | positive | 2021-07-01 | rho +0.12 | 0.22 / 1.00 | Not supported | 29% | 0.38 | |
| Pi Cycle ratio (111DMA / 2 × 350DMA) | 90d | negative | 2019-05-01 | rho +0.05 | 0.70 / 1.00 | Not supported | 8% | 0.64 | |
| Pi Cycle ratio (111DMA / 2 × 350DMA) | 365d | negative | 2019-05-01 | rho −0.21 | 0.28 / 1.00 | Not supported | 11% | 0.35 | |
| Mayer Multiple | 90d | negative | 2018-01-01 | rho +0.03 | 0.74 / 1.00 | Not supported | 6% | 0.69 | |
| Mayer Multiple | 365d | negative | 2018-01-01 | rho −0.08 | 0.58 / 1.00 | Not supported | 8% | 0.51 | |
| close / 2-year MA | 90d | negative | 2017-08-01 | rho −0.09 | 0.36 / 1.00 | Not supported | 11% | 0.36 | |
| close / 2-year MA | 365d | negative | 2017-08-01 | rho −0.53 | 0.0005 / 0.012 | Association supported | 9% | 0.07 | Why? Price / 2-year MA, 365 daysPrice / 2-year MA, 365 days The question. Across every day, did price further above its 2-year average come before weaker returns over the next year? Yes, fairly strongly (a rank correlation of −0.53, where −1 would be a perfect reverse ordering). Why not counted. The sealed test flagged it. We then found that test raises false alarms too often for metrics that change slowly: on made-up returns unrelated to the metric, it fired 9% of the time instead of 5%. Allowing for that, a result this strong arises by chance about 7% of the time, so it is not significant. It is the closest of the 24, and after allowing for running 24 tests it is far from significant. Even so. This covers only about two market cycles, and a link between a level and later returns is not a trading rule; nothing here is graded. |
| close / 200-week MA | 90d | negative | 2019-05-19 | rho −0.25 | 0.20 / 1.00 | Not supported | 5% | 0.28 | |
| close / 200-week MA | 365d | negative | 2019-05-19 | rho −0.66 | 0.0005 / 0.012 | Association supported | 13% | 0.12 | Why? Price / 200-week MA, 365 daysPrice / 200-week MA, 365 days The question. Across every day, did price further above its 200-week average come before weaker returns over the next year? Yes, fairly strongly (a rank correlation of −0.66, where −1 would be a perfect reverse ordering). Why not counted. The sealed test flagged it. We then found that test raises false alarms too often for metrics that change slowly: on made-up returns unrelated to the metric, it fired 13% of the time instead of 5%. Allowing for that, a result this strong arises by chance about 12% of the time, so it is not significant. Even so. This covers only about two market cycles, and a link between a level and later returns is not a trading rule; nothing here is graded. |
| Puell Multiple | 90d | negative | 2019-04-01 | rho −0.11 | 0.21 / 1.00 | Not supported | 12% | 0.25 | |
| Puell Multiple | 365d | negative | 2019-04-01 | rho −0.33 | 0.10 / 1.00 | Not supported | 7% | 0.11 | |
| close / stock-to-flow model | 90d | negative | 2019-03-23 | rho +0.05 | 0.69 / 1.00 | Not supported | 11% | 0.64 | |
| close / stock-to-flow model | 365d | negative | 2019-03-23 | rho +0.21 | 0.77 / 1.00 | Not supported | 13% | 0.77 | |
| Pooled bottom signals (onsets) | 365d | positive | 2017-08-01 | D +0.27 (13 onsets) | 0.028 / 0.56 | Not supported | 10% | 0.08 | Why? Pooled bottom signalsPooled bottom signals The question. We pooled the start dates of all eight bottom signals (starts within 30 days merged): 13 starts from 2018 to 2023. Was the year after better than after a typical day? On average, yes. Why not supported. Much of the gap comes from March 2020, when seven signals started at once; the four starts from Apr 2021 to May 2022 were all followed by losses. Taken alone the gap looked unusual, but after allowing for running 24 tests, and after the calibration check, it could be chance. Even so. This covers only about two market cycles, and a link between a level and later returns is not a trading rule; nothing here is graded. |
| Pooled top signals (onsets) | 365d | negative | 2017-08-01 | D −0.55 (6 onsets) | 0.077 / 1.00 | Not supported | 8% | 0.11 | Why? Pooled top signalsPooled top signals The question. We pooled the start dates of the top signals that fired: 6 starts from 2017 to 2021 (MVRV Z-score above 5, Pi Cycle, Mayer above 2.4, 5 times the 2-year average). Was the year after weaker than after a typical day? On average, yes. Why not supported. Six starts is a small sample, and a gap this size could be chance even under the sealed test. Three top signals never fired at all, and no top signal has fired since April 2021. Even so. This covers only about two market cycles, and a link between a level and later returns is not a trading rule; nothing here is graded. |
Association only: these tests ask whether a metric’s level on a day lined up with later returns. No buy/sell rule is graded here, so there is no grade column. Open Why? on a flagged or pooled row for a plain-English explanation.
Windows end 2025-09-24 (365-day horizon) or 2026-06-26 (90-day), so every day has a full forward return. Rho is Spearman’s rank correlation between the metric on a day and the log return over the next 90 or 365 days. D is the mean 365-day log return after pooled onsets minus the mean over all days. p = 0.0005 is the smallest value 2,000 draws can give. The last two columns are not preregistered and do not change the sealed verdicts; after Holm, every calibrated p is 1.00.
Calibration note (exploratory, not preregistered). The sealed null excludes shifts under 365 days. For a metric that changes slowly, the shifted copies it keeps are less extreme than the real alignment, so the test rejects too often. We did not change the sealed test; it was run once, unchanged. Instead we kept each real metric and replaced the returns with 300 surrogates that keep the returns’ own autocorrelation but are unrelated to the metric by construction (amplitude-adjusted phase randomisation), and ran the sealed test on each. The share of surrogates rejected at 5% is the real false-positive rate; ranking the observed p among the surrogate p-values gives the calibrated p. The surrogates keep the returns’ autocorrelation but not every feature of Bitcoin returns. Per-test results: family_c_calibration.json.
The same kind of shift null was used for the Family A and B zone tests. If it is lenient there too, that could only have made those results look stronger, and none of the three testable ones was significant.
Descriptive only, with hindsight. Splitting days into fifths of each metric (edges set on the whole sample, which uses hindsight), the valuation metrics pointed the way their creators claim: the average 365-day return after days in the lowest fifth vs the highest fifth was +101% vs −35% for Reserve Risk, +103% vs −31% for price / 200-week MA, +93% vs −27% for price / 2-year MA and +97% vs −20% for MVRV Z-score. For stock-to-flow it was +6% vs +84%, the opposite way. About two cycles can line up like this by chance; this table is not evidence.
Forward watch (blind, not yet evaluated). From 2026-09-25 we log every new onset of the sealed Family A and B rules, append-only, with the hash of the data used. Nothing is evaluated until a separate plan is sealed, before any forward return after a logged onset is computed. On 2026-09-24, the last day of the sealed data, four zones were on and are recorded as in progress at the start: price below the 2-year MA (since 2026-01-29), price below the stock-to-flow model (since 2021-04-18), Reserve Risk below 0.0026 (since 2022-01-21) and price above the STH cost basis (since 2026-08-19). This is a record for a later test, not a signal.
Seal, limits and disclosures
Seal. Preregistration SHA-256 aafe3c37f5bbf83bc6e4b9a986ef1c936c3b74a927c253b833a5a7626482de40; seal record f8d7d587466a4bb61b5c179aba44bb766825a6a495cc3ecb68d16e39c1810275. Family B data-source deviation SHA-256 f86c7d555e478b5f64d39b21510cc8cd1a8316cb1a3383d5b15ed3a14d2c3889, timestamped by DigiCert and FreeTSA on 2026-09-26 at 03:52:55 UTC, before any Family B metric was computed. Family C preregistration (not blind) SHA-256 e2b80b102e201ab13aaedb81033048f07f125c402df6e4a9965e6cdeb317da57, timestamped by DigiCert and FreeTSA on 2026-09-26 at 05:01:04 and 05:01:05 UTC, before any Family C statistic was computed. The preregistration, receipts, licence record and code will be published with the public toolkit, which is being prepared (samples/onchain1/). Full summary report.
Disclosures. This is not a blind test: we had already seen Bitcoin’s price path, and four of the twelve indicators (all in Family A) are functions of price alone. The protection is that every rule came from its creator and every test was fixed and sealed before the run. Our own choices, sealed in advance: “close to” the 200-week MA means within 10%; stock-to-flow uses a daily trailing-year flow; Puell bands are Glassnode’s (above 4, below 0.5); the Mayer window starts 2018-01-01 because its first-publication month is unverified. The 200-week heatmap’s colour claim was not tested because its thresholds were not published in a form we could find. Deviations: a clerical timestamp field in the sealed document is 22 seconds later than the seal itself (the RFC 3161 time is authoritative), and a loading error was fixed before any statistic was computed. Family C: the sealed text (section 7) expected a synthetic test to show roughly uniform p-values under the null. That holds for moderately persistent series but not for near-unit-root ones; we found this after the seal and before the run, ran the sealed test once, unchanged, and added the calibration check, labelled exploratory.
Family B limits. The metrics are ours, not a vendor’s. They are not entity-adjusted; days are assigned by miner timestamps; outputs are valued at the close of the day they were created. Reserve Risk and SOPR had no comparable published values to check against. In the BigQuery run, a first attempt failed on a syntax error (nothing billed) and one job’s byte cap was raised before it ran; neither affected any result, and no data fix was needed after the cross-check. We do not publish the daily metric series, because it would reveal the underlying price feed.
Prices from the Coinbase Exchange public API (and Bitstamp’s public daily closes, used only to value coins created before July 2015); we publish derived statistics only, never raw prices; no redistribution. Block headers from the public Bitcoin network; UTXO data from the public Bitcoin blockchain via Google BigQuery.
These audits analyze published indicator rules on historical data for illustration. They are not recommendations to buy, sell or hold anything, and a result is not a prediction.
Want your own backtest audited?
The same sealed rubric and tests, applied to your strategy, with a written report and fix list.
Send a high-level description. We’ll confirm scope, then send payment and secure-upload instructions.
Out of Sample · newsletter
Get new audits by email
Out of Sample, free, no spam. Every result published, including the failures.
You sign up on our newsletter page (hosted by Beehiiv); unsubscribe any time. Want a specific indicator tested? Request an audit.