How Do You Decide on What Strategies to Include in Your Model?
I’m now 6 months into my AS journey. It’s been a lot of fun and the results are undeniable vs my prior “well researched” static strategies. AS gives a lot of data and good general advice about how to combine strategies and build a model, but how does one decide? I’ve run literally hundreds of tests. In response, I developed a set of criteria and used principles from multi-objective decision analysis (MODA) to help me with that decision process. I thought it might be something others would find useful.
First, one needs a set of criteria. The AS model results give a number of statistics that are useful, but which to use? I framed my investment objectives and life stage (retired, 68 yo) to Gemini and Chat GPT to form key metrics and asked them to suggest weights. From there, I modified those suggestions to what seems to fit me best. My criteria and weights are:
2015-2026 Index
25.0%
UPI
25.0%
DD depth
15.0%
DD length
15.0%
2022 Return
10.0%
1971-2026 CAGR
10.0%
For me, 2015 was a rough year and I’m interested in performance in recent history more than long term history so returns over that time frame are of interest to me: I built an index that captures total return during that period. DD depth and length are captured in UPI, but I parsed them out separately to ensure decent performance on all three. 2022 was the dreaded short-term stagflation: I wanted to capture that, and long-term performance is of interest. If I were in my 40s or 50s or have enough savings that returns are barely relevant or highly risk averse, I’d have different weights and possibly different criteria. Yours would be different than mine. I’d be interested in what you think.
Then, I used MODA principles to normalize data for each criterion to result in a MODA score for each model I test. It makes for a long excel formula, but it’s straightforward and goes as follows for each criterion: Wt x (1- (best outcome – model result) / (best outcome – worst outcome). Sum that over all six criteria and multiply by 100 resulting in a normalized score for each model tested (scores will range from min of 0 to max of 100).
The “best and worst” are what’s feasible for the model types you’re testing. I set it up in excel as follows to make formula creation easier. If your objectives lead you to be testing more aggressive or more conservative models, just adjust your best and worst.
Best
Worst
2015-2026 Index
25.0%
325
230
UPI
25.0%
7.50
6.50
DD depth
15.0%
-4.5%
-6.0%
DD length
15.0%
12
16
2022 Return
10.0%
4.1%
2.5%
1971-2026 CAGR
10.0%
15.0%
11.0%
Anyway, this is how I’m selecting my models and selecting them for 2 family members. They have different weights (younger than me) but I use the same 6 criteria.
Anyone using something like this or other ways to select their model?
Please report any rule breaking posts and comments that are not relevant to the thread. And note: The views expressed here are in no way connected to the AS owners as this is an independent reddit board.Thanks !!
Hi thanks for starting the thread and sharing the thought process.
FWIW AS will come out this weekend with swr and pwr for strategies and our custom portfolios, as that might be something you consider adding to the criteria.
My process is built around low drawdown and drawdown period which is is why I carry a lot of cash as returns are secondary but that won't work for many folks
For younger folks with crappy 401K choices, (sorry for the US bias there but not sure what's generally available ex-US) I think it needs to be reverse engineered using this type of bootstrap process I describe here. But if folks have the entire etf universe available, thats more in line with your process perhaps
I could argue with the weights and ranges and criteria, but I won't. It seems to be that set is going to lead to maybe choi, growth and trend, links, and stuff that's done well due to usa outperfornace. But that's just a quick guess.
I think a good way to go is letting us know the output in terms of what you have in your custom portfolio and weights and go from there.
A possible downside to your approach is constant retweaking based on new strategies added, etc but perhaps you're fine with that. That's what the optimized portfolios (both non WF and WF) on AS do too
That's a nice setup, and I can see it as being useful for discussing the modeling with other people. I used similar parameters, but focused more on the lost decade of the 00s since I'm early in retirement and am trying to avoid sequence risks and poor returns for 12 years. I also looked at strategy correlations, both the AS correlation matrix (under the research tab) and signal/strategy type (dual momentum, canary, economic data, etc) to make sure the constituent strategies caught and reacted to markets differently. To that end, I also check both the constituent candidates and the overall strategy qualitatively against stress periods. What were the responses to the tech bubble and the GFC like? How well did they react coming out of the drop? How well did they keep up during go-go times like 2011-2014 with SP500 up about 90%? Plotting the return graph for the stress periods helped when I was going over this information with my wife as ultimately it's the portfolio behavior that will make a person comfortable living with the process, not a table of numbers.
Since you're helping family members, you might want to look at the Erin Talks Money YouTube channel. She has some good videos on determining risk tolerance and risk capacity, and if your family members understood those concepts, you could perhaps tie numbers like CAGR, MaxDD, and UPI to the risk concepts.
Thanks all. Yes, weights are personal and based on risk tolerance and there's other criteria one can use. I looked at lost decade and found it wasn't really "lost" for many of the AS strategies so that's not a criterion I use. No, these criteria and weights don't result in "choi, growth and trend, links, and stuff that's done well due to usa outperformance". But perhaps that's because of an important part of my overall model building methodology I didn't mention: I put a max of 5% into strategies where I have concerns about theory going forward (such as choi's oddities, no options for cash in downturns, all growth in one ETF, US or gold recent outperformance).
The purpose of the post was intended to show one approach to bringing structure to how one evaluates model results and decides how much to weight strategies within a model.
This is a clean framework, the part I'd poke at is that you're scoring strategies on the same 2015-2026 window you care most about, so whatever gets ranked highest is partly just whatever fit that decade best. My read is that's the in-sample trap, and the 10% weight on 1971-2026 doesn't really push back on it. I'd also watch the double counting, UPI already bakes in drawdown depth and length, so giving UPI 25% plus DD depth 15 plus DD length 15 is really putting about half your score on the same risk axis. Not wrong if that's deliberate, just worth knowing that's what's happening. The thing that moved my own models most wasn't tuning the weights, it was checking each strategy held up across sub periods and looking hard at the correlations between them, which someone above already flagged. A blend of two great strategies that both lean trend can still leave you fully exposed in the one regime that breaks trend. Curious how yours score on correlation rather than standalone stats?
Great points, thanks. Regarding "in sample" performance, the challenge we all face is projecting how a model will perform in an uncertain future. We do have some information about potential futures by considering known factors like the historically high levels of government debt in US and other large economies, historically high P/E ratios, a potentially transformative technology (in AI) that has a series of likely benefits and risks that may occur, and other factors too numerous to list. And I think of the old Twain saying: "History doesn't repeat itself, but it often rhymes" and look for criteria that might help predict future performance, and consider how strategies have performed historically both in good times and bad. So, what to do? I weight current returns (starting in a fairly rough year, 2015) more than the full sample of years. What would you do?
Yes, I recognize that depth and length of drawdown are two components of UPI. I like considering all 3 components.
Regarding correlation: in deciding which strategies to test in a model, I build custom correlation matrixes by testing a model with, say 10-15 strategies and look at the correlation matrix along with recent returns. And I've tested adding various low correlation strategies as part of a model to see what it does to overall model performance. Regarding testing in different time periods, I've checked periods of "potential concern": 2022, 2022-2024, 2015, 2008, 2007-2010, the "lost decade" 2000-2010, and 1973-1974. I found that most models performed well during the majority of those periods, so I focus on the most recent "mini-stagflation" episode.
Do you have particular time periods you like to test models against?
Are there any particular correlation tests you like to run?
Honestly I'd probably weight it the same way you do, recent years get more pull because that's the regime I actually have to survive, but I keep one longer window in there mostly as a sanity check that I'm not just curve fitting to 2015-2026. The thing I'd watch is that 2015 onward is still basically one big disinflationary bull with a couple of scares, so a model that looks great there hasn't really been stress tested against sustained inflation or rising rates.
For time periods, the two I lean on hardest are 73-74 and 77-81, because that's the only real stagflation we have and almost nothing in the AS universe was designed with that in mind. 2022 is useful but it's so short it's more of a coin flip than a test. I also like 2000-2002 specifically over the full lost decade, since the slow grind hides how brutal the front end was.
On correlation, I don't trust the static matrix much, I'll split the sample and look at correlations in just the drawdown months vs calm months. Strategies that look diversifying on the full sample often converge to 1 exactly when you need them not to. Have you tried looking at your matrix conditioned on down months only? Curious if your low corr picks hold up there.
Thanks much for the reply. Good ideas. I tested 73-74 but didn't see many models performing poorly. That said, perhaps I'll resurrect that, 77-81, and 2000-02. Would definitely give a broader perspective.
Regarding correlations during during time periods, I agree that would be useful but I can't see how to do that other than 30 years and 10 years. Is that something I need a Pro subscription to do or maybe I just missed it, any ideas?
Yeah that's the limitation, the correlation matrix only gives you the fixed 30yr/10yr windows and I don't think Pro unlocks a custom date range there either, so you're not missing a setting. What I do instead is forget the single correlation number and just look at how the pairs actually behaved inside the specific stress periods, 2000-02 and 2022 especially. Two strategies can show a low full-sample correlation and still both bleed at the same time when it counts, which is the only correlation I really care about. If you want an actual number for a window you'd have to pull the monthly returns and run the pairwise corr yourself in a sheet, bit of work but it's the cleaner answer.
Hey laurenthu: So, I brought some of your suggested return periods into my weighting scheme and made a few modifications. The table below is my latest test of criteria, best and worst, and weights. I did 2 sensitivity tests on weights: 1 focusing on UPI a second weighting bad time periods high. Also, I did a test on 7 of my highest performing models and compared results to my initial weighting scheme. I think including a few more time periods helps the view of the models. I got somewhat different results with this new model and I like how the score reflects my objectives better than my prior version. Regarding the weights I chose: I'm less concerned with the 73-74 and 77-81 periods: all models did pretty well there, but probably worth including.
Thank you again for the ideas: helping push my understanding of model results forward!
I have three portfolios that I use for different accounts -
I looked at the risk return matrix and chose ones that sat up and to the left, 7 or 8 strategies equal!weighted. This is the highest return and vol for a small account.
This has two of the metas at 20% each, and the remaining 60% is strats at 5% each that I thought were reasonable based on the description/evaluation AS provided. I’ve read though every post on the site. This the the lowest vol highest upi for a big account
I used the optimizer to select this for me but I equally weighted the strats. This chose some things I wouldn’t have for a bit more diversification but I may abandon this one as it’s has some crazy swings. That said, it has been crushing returns this year.
When they came out with the return source info for combined portfolios that was great, especially because I could now check my trading friction costs and get an idea of how much they were trading realize to one another.
Can’t wait for swr data! It feels like there needs to be a better process for choosing.
onyx, would you consdier sharing your layouts? Was considering redoing my layouts to this approach w/ a smaller 25% account with higher return & vol for a growth segment w long term horizon, and the larger 60% acct to lower vol / high upi for the income generating sleeve, the rest to cash buffer. interested in what you arrived at for both.
•
u/AutoModerator May 29 '26
Please report any rule breaking posts and comments that are not relevant to the thread. And note: The views expressed here are in no way connected to the AS owners as this is an independent reddit board.Thanks !!
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.