A trend-following strategy that looks profitable in backtesting is not necessarily a profitable strategy. It might be overfit to the historical period you tested it on. It might be capturing a risk premium that existed in 2010 and has since been arbitraged away. It might simply be an artefact of having tested enough parameter combinations that one was bound to look good by chance. Statistical rigour is the only way to distinguish a genuine edge from a well-dressed accident.
In-sample testing is not testing
The most common mistake: develop a strategy, optimise its parameters on historical data, report the performance on that same data. This is not a backtest. It's a curve fit. The parameters are optimised for the noise in the historical period as much as for any genuine signal. The reported performance will almost always be better than live performance.
The minimum viable alternative: split your data into a development set and a test set. Develop and optimise on the development set. Run the final strategy once — exactly once — on the test set and report those numbers. If you touch the test set during development, it becomes part of the development set and you need a new holdout.
Walk-forward testing
A single train/test split is better than nothing but still vulnerable to the choice of split date. Walk-forward testing removes that dependency by rolling the window forward through time:
def walk_forward(prices, strategy_fn, train_months=24, test_months=6):
results = []
start = prices.index[0]
end = prices.index[-1]
cursor = start + pd.DateOffset(months=train_months)
while cursor + pd.DateOffset(months=test_months) <= end:
train = prices[start:cursor]
test = prices[cursor:cursor + pd.DateOffset(months=test_months)]
# Optimise params on train, apply to test
params = strategy_fn.optimise(train)
out_of_sample = strategy_fn.run(test, params)
results.append(out_of_sample)
cursor += pd.DateOffset(months=test_months)
return pd.concat(results)
The equity curve from a walk-forward test is stitched together from out-of-sample periods only. It's a conservative estimate of real-world performance, which is exactly what you want before risking capital.
Parameter sensitivity
A robust strategy should perform reasonably across a range of nearby parameter values, not only at the single optimised point. If performance degrades sharply when you shift a moving average period from 20 to 19 or 21, the strategy is fragile. Test a grid of parameters and look at the performance surface — a robust edge will produce a smooth hill, not a single spike surrounded by underperformance.
Transaction costs are not optional
Test with realistic costs applied: commission per trade, estimated slippage (typically modelled as a fixed percentage of the spread), and financing costs for leveraged positions. For a trend-following strategy trading daily, even a 0.05% round-trip cost can consume the majority of a strategy's edge. If the strategy's Sharpe ratio collapses when costs are added, the gross edge is real but the net edge is not.