The assumption nobody measured
One unmeasured assumption moved this backtest result by $111,344.
What the video says
One unmeasured assumption moved this backtest result by $111,344.
Same trend signal. 8 futures markets.
Backtest, 2020–2026.
In the backtest, before trading costs:
ZN gained +$122,656.
Assume 1 tick of slippage every time it buys or sells.
ZN's backtest: +$58,077.
Now assume 3 ticks.
ZN's backtest: −$53,267.
That swing is 91% of everything ZN gained before costs. ES, GC and NQ each change under 10%.
Why is ZN's result so sensitive?
A typical ZN trading day moves 26 ticks. The other markets: 182–983.
Under a flat-tick cost model, the fewer ticks a market moves, the larger each tick of slippage is relative to its risk.
But slippage here was ESTIMATED, not measured.
Bid/ask data: MISSING. And neither of ZN's results is statistically distinguishable from zero.
ALPHA-1A-EXP6-TREND-BASE
Backtest on $5M of hypothetical research capital. Research conclusion: ALPHA-1A INCONCLUSIVE. Not investment advice. Manifest 323828d4887c.
Evidence behind it
- SlippageESTIMATED
- Bid/ask spreadMISSING
- CommissionESTIMATED
- Contract specsCONFIRMED
Marks open only as wide as each input’s evidence class allows. See Method.
Proof card
Experiment: ALPHA-1A-EXP6-TREND-BASE · manifest 323828d4887c Sealed research conclusion: ALPHA-1A INCONCLUSIVE · investment conclusion: NO PROMOTION
Hypothesis
Under a flat ticks-per-side slippage model, a market whose tick is large relative to its typical daily move will have its backtest result dominated by the slippage assumption rather than by its signal.
Evidence
- Same trend signal, with the experiment's 100-day lookback as defined in its sealed manifest, applied to 8 futures markets, 2020–2026.
- ZN gross result before costs: +$122,656.
- Slippage is charged per side, meaning on every buy and every sell. A roll is charged as a full close and reopen, each leg at the full per-side cost, where per-side cost is commission plus slippage as the experiment's cost model defines it.
- At 1 tick per side: ZN nets +$58,077.
- At 3 ticks per side: ZN nets −$53,267.
- The swing between the two assumptions is $111,344, or 91% of the gross result.
- ES, GC and NQ were profitable at the low assumption and changed by less than 10%.
- YM and ZN change sign between the two assumptions. No market's result is statistically distinguishable from zero at either assumption (see Statistical weight).
In-sample vs holdout (ZN, net)
| Slippage | In-sample | Holdout |
|---|---|---|
| 1 tick | +$66,104 | −$8,028 |
| 3 ticks | −$37,864 | −$15,402 |
The positive low-slippage total comes entirely from the in-sample period. In the holdout, ZN lost money under both assumptions.
Statistical weight
- ZN: t ≈ 0.31 at low slippage, −0.29 at high. Neither result is statistically distinguishable from zero (|t| < 2).
- No market's result reaches |t| ≥ 2 at either level; the largest is GC at 1.78.
Method
Sealed experiment outputs, read by pinned blob hash; daily P&L reconciled to the sealed scorecards for every market at both slippage levels, and period sums reconciled to totals.
Diagnostics (post-hoc, not preregistered):
- Typical daily move: ZN moves 26 ticks on a typical weekday session, against 182–983 for the other markets. Counting all rows including Sundays gives 24 for ZN; the ranking does not change.
- Significance: t = mean over standard error of daily net P&L from the first position onward. It ignores autocorrelation, so it is approximate.
Evidence classes of the inputs
- Slippage: ESTIMATED (flat placeholder, not measured)
- Bid/ask spread: MISSING
- Commission: ESTIMATED
Failures and limits
- Slippage was never measured. A flat-tick model penalizes coarse-tick markets by construction, so part of this result may reflect the cost model's shape.
- Neither ZN result is statistically meaningful, so the swing shows sensitivity to an assumption, not a change in a real edge.
- The explanatory diagnostics were computed after the fact.
- The daily rows include Sundays, and a post-closure finding logged for this experiment confirmed that it treats them as trading days: its lookback and volatility windows count rows, and sizing and annualized figures scale per row. Positions ran larger than designed for most markets, and not by the same amount for every market.
- What that changes: the absolute dollar figures on this card come from the sealed results as recorded and are affected by the sizing error.
- What it mostly doesn't: the error mostly rescales positions, so ratios such as the swing's share of the gross result, and the t-statistics, are expected to be largely unchanged. The experiment was not rerun, so this is an expectation, not a measurement.
- The research conclusion is expected to stand.
- Backtest on $5M of hypothetical research capital; no live execution.
Out-of-sample / forward test
Holdout split shown above. No forward test.
Conclusion
SUPPORT WITH CONDITIONS, for a methodological claim only: in this experiment the unmeasured slippage assumption moves ZN's result by an amount comparable to its entire gross result. No claim is made that ZN is or is not tradeable with this signal.
Confidence
Not scored. The deciding input is classified ESTIMATED.
Not investment advice.
Provenance
- resolved story sha256
c08b86d2e51da64902a9a0bbcd3674da910101e9240bfa90c7d7921c0f6ce811- rendered video sha256
9e79f17e5206488d43ef458ede63ecc02a59bd7af4d720b61b779f22dda2d701- web video sha256
41495f65bd81c17a4c6a2afb1b4e5838357ac853fed8232033a936b9732117d2- experiment id
ALPHA-1A-EXP6-TREND-BASE- manifest hash
323828d4887cdc4a2490fb02be15cc350e0e4606f0e080ce93236fbc87432ccf- source blobs · daily_pnl
903b27d37c51be00031f95ec1a52ceeb55f40323- source blobs · results
da2c61f0c464856a6a686c7b3c092516f79b7857- source blobs · manifest
4f440afb95a610c4ec326bef8699d6686f105afd- source blobs · post_closure
e73451b66656f6deaa0248e9a3d7431a8a05f3c0- source blobs · costs_py
aee1671ba0a44f6e612ac676fdc65035c6b6c0b8
Backtests and model estimates on historical data. Not investment advice.