The past is not a good guide to make predictions about the future because the future will be different. One should take as little as possible but as much as necessary from a backtest. A backtest shows how an idea would have worked out in the past if one followed a specific set of rules. It doesn’t tell anything about its performance going forward. People use backtesting tools to come up with a strategy. This is called fitting the data or curve-fitting. It often falls apart in live trading or just gives miserable results.
It’s so important, so let me tell it again. Backtesting can’t be used to forecast the future and it can’t be used to come up with a strategy. Backtesting is only to validate the statistical significance of a strategic idea on historic data.
This is how it usually works out for new people getting into quant trading:
Photo by PyQuant News on LinkedIn: source
Even if we don’t over optimize nor curve-fit the strategy to look good on past data, we still use the past data and whenever we backtest anything on historical prices, it’s going to be a fit of some sort on that price data. That’s why one needs to be extremely mindful when doing research with past price data. It also applies to all the “when this happened, then this will happen next” type of charts or “the man who saw the ’08 crash says this will happen” shared on social media and news. Most of the time it’s random BS in a statistical sense. One of the main reasons is low sample size. If the market fell on two Mondays straight, does it mean that the market is going to fall on Mondays?!
How to think about backtesting the right way
Backtesting gives a better understanding of risk and trade management, drawdowns and recovery, compounding capital over time, how an edge plays out, why a strategy makes money, how an account can blow up with too much risk or leverage etc. I’ve done enough backtesting to understand a new strategy by just looking at its technical components and trade mechanics. It’s all about adjusting risk vs reward, entry / exit mechanics, position sizing, diversification, which combined together make up the trade distribution and return profile for a strategy. So if I think of a new strategic idea, I can first close my eyes and already imagine the trade distribution and return profile for that particular style without running a backtest software. Of course, I should still do that to validate my thoughts, but the main point is not the exact result or profit from a backtest, but how to combine different edges and manage risk at the portfolio level using a basket of uncorrelated strategies, without knowing what the future path will look like.
My job as a trader is to put odds in my favor and manage risk. It’s not what most people think that is to sit in front of screens and try to predict the next move or a winning stock. It’s just pure math and working with data. The market hours are only for executing a predetermined trading plan. The market hours are not for trying to outsmart the next candlestick on a chart.
The stock market is an ever-changing environment
I think the market has technically changed a lot in the past 5 years. You just can’t look at short-term candlestick patterns on a chart and expect it to show investors’ behavior. Most of the trading done today is by bots and algos. A lot of action is in the huge derivatives markets, where market makers hedge their books throughout the day, so the charts show a random distribution of trades created by sell and buy orders in the book. We may see a heavy sell-off one day, a strong rally the next, or the market tanking into close after a quiet sideways day, and things don’t often make sense. I accept it and consider charts to be just a visual representation of data, which most of it is noise in short-term anyway. The idea behind a market index like S&P 500 used to be that the index follows stocks like AAPL, MSFT etc, but today it seems that AAPL and MSFT are following the index trading, at least in the shorter term space. It’s like the tail is wagging the dog. We’ll see how extreme it can go and what kind of effect it may have on the market.
Share this post