Breaking News
Investing Pro 0
Last Call for Cyber Monday! Save Now on Claim 60% OFF

3 Ways to Test Your Investment Strategy

By James PicernoMarket OverviewAug 05, 2023 10:01
ng.investing.com/analysis/3-ways-to-test-your-investment-strategy-178690
3 Ways to Test Your Investment Strategy
By James Picerno   |  Aug 05, 2023 10:01
Saved. See Saved Items.
This article has already been saved in your Saved Items
 
 
US500
+0.56%
Add to/Remove from a Portfolio
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
AGG
+0.82%
Add to/Remove from a Portfolio
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 
SPY
+0.58%
Add to/Remove from a Portfolio
Add to Watchlist
Add Position

Position added successfully to:

Please name your holdings portfolio
 

There are three basic ways to test an investment strategy. Each has its own unique set of pros and cons, but only one is practical.

1. Out-Of-Sample Testing With Real Money

The best approach is to develop a strategy and then run it with real money out of sample for at least 3 to 5 years. Longer is even better. That’s the gold standard, but that takes time, and so there are obvious limitations. Be careful not to confuse this version of out-of-sample testing with its pseudo-out-of-sample cousin, which uses a portion of historical data to build a model and then tests it on the remaining unused “out of sample” historical numbers. Useful, but no substitute for the genuine article.

2. Paper Trading

The weakest alternative is to develop a strategy and paper trade to decide if it passes the smell test. Relatively easy and quick, but here too, there are clear challenges, namely, the transition from theory to empirical usually brings many surprises.

3. Backtesting

The best (or should we say the least worst) alternative is to backtest a strategy. The idea here is that you can have the best of both worlds: a rough approximation in the here and now of how a strategy would have fared if implemented at some point decades ago. Alas, this is no silver bullet either since no backtest can flawlessly tell you how a strategy will perform in the years ahead. But short of acquiring the powers to see into the future, it’s the best that mere mortals can do.

Indeed, the key advantage to historical backtesting: you don’t have to wait years to determine if a strategy is a winner or a dog. Another plus: you’re not totally reliant on theory for assessing how the future may unfold.

The critical issue, of course, is designing a backtest that comes close to replicating the real world through a historical lens. Easier said than done. A poorly designed backtest is usually worse than simply making guesstimates. That’s a crucial hazard since there are more ways to delude yourself with backtests than there are techniques for developing a robust test.

In fact, building a useful backtest is a delicate dance of art and science. Ideally, you’ll run many techniques, recognizing that developing useful backtest data and assessing it accurately and objectively is a bit like the tale of blind men trying to describe an elephant. Approximating the truth requires combining multiple descriptions and perspectives.

In short, there are no silver bullets for building a solid backtest. Quite a bit of successfully building and evaluating historical simulations is avoiding rookie mistakes. One error I see a lot is using a single time window to do the heavy lifting.

For instance, backtesting a strategy that looks impressive over a 2000-2023 sample period may be misleading because it relies heavily on sidestepping much of the 2008-2009 financial crash. But it’s problematic if, after taking out that period or using a post-2009 start date, the strategy falls apart.

The Best Way to Backtest a Strategy

There are many ways to guard against this pitfall, including my favorite technique: assembling a backtest using rolling-forward start dates and then assessing all the time-window results for determining the strategy’s stability (or lack thereof) through time.

As an example, consider a simple 60%/40% stock/bond portfolio that’s rebalanced to the target weights at the end of each calendar year. We’ll use SPDR® S&P 500 (NYSE:SPY) and iShares Core U.S. Aggregate Bond ETF (NYSE:AGG). This toy example begins with a Jan. 1, 2016 start date and calculates the annualized return through Aug. 2, 2023, via daily numbers.

The analytics calculates the full period return using a Jan. 2, 2016 start date and then uses a Jan. 3 start date, and so on. The goal is to aggregate all the annualized returns for each time window and evaluate the distribution, as shown in the chart below.

Distribution Of 60/40 Strategy's Annualized Returns
Distribution Of 60/40 Strategy's Annualized Returns

Conclusion

The main takeaway: the performance is heavily skewed toward a moderately positive result. The interquartile range of returns: is 3.6% to 6.8%, shown by the two blue lines, with a median of 6.2% (red line). Deciding if this is acceptable or not is a bigger question. The point, for now, is that we’re not relying on one time window, which may be deeply flawed for one reason or another.

If this was a robust rolling-forward backtest, we’d use a much earlier start date. We’d also run a batter of other analytics before making a final judgment. But as a first step for deciding whether it’s wise to go deeper or look elsewhere, this is a relatively painless, quick, and useful test. By contrast, a backtest that uses one start date may be faulty in the extreme.

3 Ways to Test Your Investment Strategy
 

Related Articles

3 Ways to Test Your Investment Strategy

Add a Comment

Comment Guidelines

We encourage you to use comments to engage with users, share your perspective and ask questions of authors and each other. However, in order to maintain the high level of discourse we’ve all come to value and expect, please keep the following criteria in mind: 

  • Enrich the conversation
  • Stay focused and on track. Only post material that’s relevant to the topic being discussed.
  • Be respectful. Even negative opinions can be framed positively and diplomatically.
  •  Use standard writing style. Include punctuation and upper and lower cases.
  • NOTE: Spam and/or promotional messages and links within a comment will be removed
  • Avoid profanity, slander or personal attacks directed at an author or another user.
  • Don’t Monopolize the Conversation. We appreciate passion and conviction, but we also believe strongly in giving everyone a chance to air their thoughts. Therefore, in addition to civil interaction, we expect commenters to offer their opinions succinctly and thoughtfully, but not so repeatedly that others are annoyed or offended. If we receive complaints about individuals who take over a thread or forum, we reserve the right to ban them from the site, without recourse.
  • Only English comments will be allowed.

Perpetrators of spam or abuse will be deleted from the site and prohibited from future registration at Investing.com’s discretion.

Write your thoughts here
 
Are you sure you want to delete this chart?
 
Post
Post also to:
 
Replace the attached chart with a new chart ?
1000
Your ability to comment is currently suspended due to negative user reports. Your status will be reviewed by our moderators.
Please wait a minute before you try to comment again.
Thanks for your comment. Please note that all comments are pending until approved by our moderators. It may therefore take some time before it appears on our website.
 
Are you sure you want to delete this chart?
 
Post
 
Replace the attached chart with a new chart ?
1000
Your ability to comment is currently suspended due to negative user reports. Your status will be reviewed by our moderators.
Please wait a minute before you try to comment again.
Add Chart to Comment
Confirm Block

Are you sure you want to block %USER_NAME%?

By doing so, you and %USER_NAME% will not be able to see any of each other's Investing.com's posts.

%USER_NAME% was successfully added to your Block List

Since you’ve just unblocked this person, you must wait 48 hours before renewing the block.

Report this comment

I feel that this comment is:

Comment flagged

Thank You!

Your report has been sent to our moderators for review
Continue with Google
or
Sign up with Email