
Please try another search
There are three basic ways to test an investment strategy. Each has its own unique set of pros and cons, but only one is practical.
The best approach is to develop a strategy and then run it with real money out of sample for at least 3 to 5 years. Longer is even better. That’s the gold standard, but that takes time, and so there are obvious limitations. Be careful not to confuse this version of out-of-sample testing with its pseudo-out-of-sample cousin, which uses a portion of historical data to build a model and then tests it on the remaining unused “out of sample” historical numbers. Useful, but no substitute for the genuine article.
The weakest alternative is to develop a strategy and paper trade to decide if it passes the smell test. Relatively easy and quick, but here too, there are clear challenges, namely, the transition from theory to empirical usually brings many surprises.
The best (or should we say the least worst) alternative is to backtest a strategy. The idea here is that you can have the best of both worlds: a rough approximation in the here and now of how a strategy would have fared if implemented at some point decades ago. Alas, this is no silver bullet either since no backtest can flawlessly tell you how a strategy will perform in the years ahead. But short of acquiring the powers to see into the future, it’s the best that mere mortals can do.
Indeed, the key advantage to historical backtesting: you don’t have to wait years to determine if a strategy is a winner or a dog. Another plus: you’re not totally reliant on theory for assessing how the future may unfold.
The critical issue, of course, is designing a backtest that comes close to replicating the real world through a historical lens. Easier said than done. A poorly designed backtest is usually worse than simply making guesstimates. That’s a crucial hazard since there are more ways to delude yourself with backtests than there are techniques for developing a robust test.
In fact, building a useful backtest is a delicate dance of art and science. Ideally, you’ll run many techniques, recognizing that developing useful backtest data and assessing it accurately and objectively is a bit like the tale of blind men trying to describe an elephant. Approximating the truth requires combining multiple descriptions and perspectives.
In short, there are no silver bullets for building a solid backtest. Quite a bit of successfully building and evaluating historical simulations is avoiding rookie mistakes. One error I see a lot is using a single time window to do the heavy lifting.
For instance, backtesting a strategy that looks impressive over a 2000-2023 sample period may be misleading because it relies heavily on sidestepping much of the 2008-2009 financial crash. But it’s problematic if, after taking out that period or using a post-2009 start date, the strategy falls apart.
There are many ways to guard against this pitfall, including my favorite technique: assembling a backtest using rolling-forward start dates and then assessing all the time-window results for determining the strategy’s stability (or lack thereof) through time.
As an example, consider a simple 60%/40% stock/bond portfolio that’s rebalanced to the target weights at the end of each calendar year. We’ll use SPDR® S&P 500 (NYSE:SPY) and iShares Core U.S. Aggregate Bond ETF (NYSE:AGG). This toy example begins with a Jan. 1, 2016 start date and calculates the annualized return through Aug. 2, 2023, via daily numbers.
The analytics calculates the full period return using a Jan. 2, 2016 start date and then uses a Jan. 3 start date, and so on. The goal is to aggregate all the annualized returns for each time window and evaluate the distribution, as shown in the chart below.
The main takeaway: the performance is heavily skewed toward a moderately positive result. The interquartile range of returns: is 3.6% to 6.8%, shown by the two blue lines, with a median of 6.2% (red line). Deciding if this is acceptable or not is a bigger question. The point, for now, is that we’re not relying on one time window, which may be deeply flawed for one reason or another.
If this was a robust rolling-forward backtest, we’d use a much earlier start date. We’d also run a batter of other analytics before making a final judgment. But as a first step for deciding whether it’s wise to go deeper or look elsewhere, this is a relatively painless, quick, and useful test. By contrast, a backtest that uses one start date may be faulty in the extreme.
U.S. stocks ended November on a high note as cooling inflation boosted hopes the Fed is done hiking. December is expected to be another strong month as per recent history. Looking...
I previously discussed a slate of recessionary indicators with high correlations to recessionary onsets. However, as we head into 2024, many Wall Street economists predict a “soft...
Financial “wisdom” is said to be cyclical rather than cumulative, but that’s unfair. At least in the dominion of portfolio management and design, academics and money managers have...
Are you sure you want to block %USER_NAME%?
By doing so, you and %USER_NAME% will not be able to see any of each other's Investing.com's posts.
%USER_NAME% was successfully added to your Block List
Since you’ve just unblocked this person, you must wait 48 hours before renewing the block.
I feel that this comment is:
Thank You!
Your report has been sent to our moderators for review
Add a Comment
We encourage you to use comments to engage with users, share your perspective and ask questions of authors and each other. However, in order to maintain the high level of discourse we’ve all come to value and expect, please keep the following criteria in mind:
Perpetrators of spam or abuse will be deleted from the site and prohibited from future registration at Investing.com’s discretion.