Why the testing lesson matters
The concept lesson explains the logic. This lesson asks whether that logic survives contact with actual rules, costs and failure conditions. The first discipline is to freeze the strategy before looking at the result. If entry, exit, sizing or exceptions are edited after a loss, the test is no longer evaluating the same strategy.
A useful strategy lab does not ask only “did it make money?” It asks whether the rule was executable, whether the costs were modeled honestly, whether the sample represented more than one favorable market regime, and whether the losses came from normal variance or from an assumption that no longer held.
Freeze the rule set
Write the strategy in a table before testing:
| Rule component | What must be fixed before the test |
|---|---|
| Market / universe | Which assets and venues are eligible, and why |
| Timeframe | Data interval and decision time |
| Entry | Exact observable condition that creates a position |
| Position size | How risk or allocation is calculated |
| Exit / invalidation | What closes or reduces the position |
| Costs | Fees, spread, slippage and other relevant friction |
| No-trade condition | When the signal is ignored even if it appears |
| Review trigger | Evidence that causes investigation rather than ad-hoc editing |
For backtesting, the most important assumptions from the paired lesson should appear explicitly in this table. If an assumption cannot be observed or tested, label it as judgment rather than pretending it is mechanical.
Build the cost and friction ledger
Start with gross strategy outcome and then subtract the costs created by the way the strategy actually trades. A simple educational ledger is:
Net result = Gross trading result − explicit fees − estimated spread/slippage − financing or transfer costs − other strategy-specific friction.
Not every family has every cost. Long-term investing may have low turnover but meaningful custody and conversion considerations. Arbitrage may be dominated by execution and transfer costs. Market making may depend on queue position, adverse selection and inventory. The purpose is to model the costs that belong to this strategy rather than paste one generic fee assumption across all styles.
Stress the assumption that matters most
Backtests fail through look-ahead bias, survivorship bias, unrealistic fills, ignored delistings, missing fees, overfitting and repeated testing on the same “out-of-sample” period. They also fail when the trader treats historical profitability as proof that the market mechanism will persist.
A good backtest produces questions for forward testing. It does not end the research process.
Turn that into at least three stress cases: normal, unfavorable but plausible, and assumption failure. The third case is especially important. A strategy should not survive every scenario by definition. If no observation can make the method invalid, the rule is belief rather than a testable strategy.
Run the family-specific strategy lab
Write one complete strategy rule set before opening the historical outcome. Split the data into development and untouched evaluation periods. Record trade count, win rate, average win, average loss, expectancy, drawdown, turnover and cost assumptions. Then run a parameter-sensitivity check and remove the best trade. If the strategy survives only one narrow setting or one exceptional winner, label it fragile rather than “proven.”
After the first pass, change one assumption at a time. Increase costs, worsen entry quality, remove the best trade, or test a different regime. Do not optimize until the original result disappears; the aim is to understand sensitivity. A robust idea should usually make sense across a reasonable neighborhood of assumptions, even if the exact result changes.
Separate strategy failure from trader failure
When a test disappoints, classify the problem before changing anything:
- Rule failure: the strategy did exactly what it was designed to do, but the edge was insufficient.
- Execution failure: the signal had value but realistic costs, delay or liquidity destroyed it.
- Regime mismatch: the strategy was used in conditions it was not designed for.
- Process failure: the tester changed rules, skipped trades or used information unavailable at the time.
- Insufficient evidence: the sample is too small or too concentrated to support a conclusion.
Those categories lead to different next steps. Treating all losses as “bad strategy” prevents learning; treating all losses as “bad luck” prevents accountability.
Define the decision before the next sample
End the worksheet with one of four states: continue testing, investigate, modify as a new version, or retire / do not use. If you modify a material rule, give the strategy a new version and restart the relevant evidence trail. Do not blend the new rules into the old backtest as if they had always existed.
Skill check — no money needed
You can complete this lesson entirely with historical, fictional or paper data. The skill is demonstrated when another reader can reproduce the rule, recalculate the costs, see the same failure conditions and understand why the final decision was made. Profitability is not required for the exercise to be successful; discovering that a strategy does not survive realistic conditions is useful knowledge.
*Cryptocurrency and virtual asset transactions are highly volatile and irreversible, may result in significant losses, and do not guarantee returns; customers should trade only after understanding the risks involved.
