Why you should know this
One perfect historical chart is a story. A backtest asks whether the rule behaved consistently across many observations. But testing many rules and keeping only the winner can manufacture an illusion.
The purpose is not to prove profitability. It is to challenge the idea before money is exposed.
1. Write the hypothesis
Example:
On completed daily BTC/USD candles from a defined source, does a close above a 20-day high with relative volume above 1.5 produce positive forward returns over ten days, after stated costs, more consistently than a simple benchmark?
The sentence defines asset, timeframe, trigger, feature, horizon, cost and comparison. Avoid “Does this breakout strategy work?”
2. Freeze every rule
Specify:
- universe and membership dates;
- data source, time zone and adjustment;
- signal calculation and warm-up;
- execution time and price assumption;
- position size and overlap rule;
- exits and stop treatment;
- fees, spread, slippage and funding;
- missing data and venue failure;
- benchmark and metrics.
If discretion remains, log the decision rule so another reviewer can replicate it.
3. Prevent look-ahead
Look-ahead bias occurs when the test uses information unavailable at the decision time. Examples:
- entering at the same candle’s close after using that close to confirm;
- using future pivots to mark current swings;
- selecting today’s largest assets for a historical universe;
- using revised data as if known originally.
Shift signals and execution correctly. A completed signal may be executable only on the next available quote.
4. Control survivorship and delisting
A dataset containing only surviving tokens removes failures. Historical universe membership, delistings, redenominations, forks and halted markets matter.
Decide how the test handles an asset that becomes untradeable. Ignoring it usually flatters results.
5. Model costs honestly
Include maker/taker fees, spread, slippage, funding, borrow and transfer costs relevant to the rule. Slippage should vary with size, liquidity and volatility when possible.
If only candle data are available, say the model cannot reproduce intrabar order sequence. When stop and target both occur in one candle, use a conservative or explicitly randomized rule—not the favorable sequence.
6. Split research and test data
Use a development sample for creating the rule and an untouched out-of-sample period for evaluation. Time-series data should generally preserve chronology.
After inspecting the test result, it is no longer untouched. Repeatedly adjusting and retesting leaks information.
7. Record every experiment
Backtest overfitting grows when many indicators, assets, windows and thresholds are tried. Keep an experiment register:
- hypothesis ID;
- parameters;
- date run;
- reason for change;
- result—including failures;
- whether test data were viewed.
The number of failed attempts is part of the evidence.
8. Evaluate more than return
Review:
- sample size and exposure time;
- average/median trade;
- win/loss distribution;
- maximum drawdown under methodology;
- turnover and costs;
- tail losses;
- performance by regime, asset and year;
- sensitivity to nearby parameters;
- benchmark comparison.
A strategy that works only at period 17 but collapses at 16 and 18 may be fragile.
9. Paper observe
After research, run the fixed rule prospectively without money. This reveals data delays, operational issues and behavioral difficulties that history cannot.
Paper results are still not live results: fills, emotions and market impact differ.
A minimal backtest card
| Section | Required evidence |
|---|---|
| Hypothesis | Exact conditional question |
| Data | Source, rights, time, universe, gaps |
| Signal | Formula, lag, parameters |
| Execution | Fill model and costs |
| Validation | Development/test split and experiment count |
| Metrics | Return, risk, turnover, robustness |
| Limitations | Biases and unmodeled risks |
| Decision | Reject, revise before new test, or paper observe |
Common mistakes
- Selecting only beautiful examples.
- Entering before the signal was knowable.
- Ignoring failed or delisted tokens.
- Assuming candle high/low reveals intrabar sequence.
- Excluding fees and slippage.
- Testing hundreds of variants and reporting one.
- Treating paper performance as guaranteed live performance.
A no-money backtest lab
Use a fictional 100-row dataset. Define a simple moving-average rule, reserve the last 30 rows, and calculate signals with a one-period execution lag. Add fixed fictional costs.
Record every parameter tried. Test the untouched rows once. Write why the sample is too small for a live conclusion and what new evidence would be needed.
How this connects to market mastery
Backtesting turns technical language into accountable research. It connects formulas, data quality, execution and risk. Mastery is not the smoothest equity curve; it is a process that preserves failed tests and remains skeptical after success.
Key takeaways
- Start with a precise hypothesis and frozen rules.
- Prevent look-ahead and survivorship bias.
- Include costs and conservative intrabar assumptions.
- Preserve an untouched chronological test.
- Record all experiments to expose overfitting.
Completion check: Produce a reproducible fictional backtest card and identify at least five reasons it may not transfer to live trading.
*Cryptocurrency and virtual asset transactions are highly volatile and irreversible, may result in significant losses, and do not guarantee returns; customers should trade only after understanding the risks involved.
