Why the testing lesson matters
The concept lesson explains the logic. This lesson asks whether that logic survives contact with actual rules, costs and failure conditions. The first discipline is to freeze the strategy before looking at the result. If entry, exit, sizing or exceptions are edited after a loss, the test is no longer evaluating the same strategy.
A useful strategy lab does not ask only “did it make money?” It asks whether the rule was executable, whether the costs were modeled honestly, whether the sample represented more than one favorable market regime, and whether the losses came from normal variance or from an assumption that no longer held.
Freeze the rule set
Write the strategy in a table before testing:
| Rule component | What must be fixed before the test |
|---|---|
| Market / universe | Which assets and venues are eligible, and why |
| Timeframe | Data interval and decision time |
| Entry | Exact observable condition that creates a position |
| Position size | How risk or allocation is calculated |
| Exit / invalidation | What closes or reduces the position |
| Costs | Fees, spread, slippage and other relevant friction |
| No-trade condition | When the signal is ignored even if it appears |
| Review trigger | Evidence that causes investigation rather than ad-hoc editing |
For market making, the most important assumptions from the paired lesson should appear explicitly in this table. If an assumption cannot be observed or tested, label it as judgment rather than pretending it is mechanical.
Build the cost and friction ledger
Start with gross strategy outcome and then subtract the costs created by the way the strategy actually trades. A simple educational ledger is:
Net result = Gross trading result − explicit fees − estimated spread/slippage − financing or transfer costs − other strategy-specific friction.
Not every family has every cost. Long-term investing may have low turnover but meaningful custody and conversion considerations. Arbitrage may be dominated by execution and transfer costs. Market making may depend on queue position, adverse selection and inventory. The purpose is to model the costs that belong to this strategy rather than paste one generic fee assumption across all styles.
Stress the assumption that matters most
Market making fails when quotes are stale, inventory limits are ignored, volatility expands faster than quote width, or fees and cancel costs erase the spread. It can also fail operationally through API outages, latency, incorrect market data or one-sided execution.
The most important stress test is not an average day. It is the moment when spreads widen, inventory is already imbalanced and the system’s ability to cancel or hedge is impaired.
Turn that into at least three stress cases: normal, unfavorable but plausible, and assumption failure. The third case is especially important. A strategy should not survive every scenario by definition. If no observation can make the method invalid, the rule is belief rather than a testable strategy.
Run the family-specific strategy lab
Create a fictional quote simulator for twenty price updates. Record bid, ask, fills, inventory, mark-to-market P&L and spread P&L separately. Add one sudden 3% price jump and a temporary inability to cancel quotes. Then set a maximum inventory and a volatility threshold that forces quoting to pause. The lesson is complete when the reader can explain why gross spread capture is not the same as strategy profit.
After the first pass, change one assumption at a time. Increase costs, worsen entry quality, remove the best trade, or test a different regime. Do not optimize until the original result disappears; the aim is to understand sensitivity. A robust idea should usually make sense across a reasonable neighborhood of assumptions, even if the exact result changes.
Separate strategy failure from trader failure
When a test disappoints, classify the problem before changing anything:
- Rule failure: the strategy did exactly what it was designed to do, but the edge was insufficient.
- Execution failure: the signal had value but realistic costs, delay or liquidity destroyed it.
- Regime mismatch: the strategy was used in conditions it was not designed for.
- Process failure: the tester changed rules, skipped trades or used information unavailable at the time.
- Insufficient evidence: the sample is too small or too concentrated to support a conclusion.
Those categories lead to different next steps. Treating all losses as “bad strategy” prevents learning; treating all losses as “bad luck” prevents accountability.
Define the decision before the next sample
End the worksheet with one of four states: continue testing, investigate, modify as a new version, or retire / do not use. If you modify a material rule, give the strategy a new version and restart the relevant evidence trail. Do not blend the new rules into the old backtest as if they had always existed.
Skill check — no money needed
You can complete this lesson entirely with historical, fictional or paper data. The skill is demonstrated when another reader can reproduce the rule, recalculate the costs, see the same failure conditions and understand why the final decision was made. Profitability is not required for the exercise to be successful; discovering that a strategy does not survive realistic conditions is useful knowledge.
*Cryptocurrency and virtual asset transactions are highly volatile and irreversible, may result in significant losses, and do not guarantee returns; customers should trade only after understanding the risks involved.
