Diversify. This is the advice that is repeated time and again in the world of investment. Now, what does true diversification involve? Could we talk about diversification by trading a single asset with a single trading system? In this article, we’ll show you how to do it. Specification Risk: One more reason to diversify your systems.
For a while now, I have made a fierce defense of the benefits of diversification, beyond asset diversification. But it is just now, after the big fall and the subsequent recovery (I am writing this with the SP500 trading above 2900 points) when people review their systems, contradict themselves, and regret not having diversified further. It is the great movements of the market that shake our beliefs and cause us to thoroughly review our mistakes and successes. And not having diversified well, may have been one of the great mistakes of the vast majority of investors.
Two different entry points for the same system could lead to very different results over time. We must therefore also diversify the entry points in order to get as close as possible to the expected outcome of the investment system.
This point of diversification is particularly important in systems where binary decisions are made. That is, in those models in which you are invested, or not, is a type of asset depending on a signal. Many Asset Allocation and trend systems follow this philosophy.
In order to illustrate the great dispersions that we can see, in relation to the chosen points of entry, we will analyze the behaviors during this year of a very simple trend system with 2 ETFs. If for a given period the SPY has positive momentum, it will invest in the SPY, if not, in TLT. Every four weeks, we re-measure the signal and make a decision. This system represents in a simplified way the investment in equities or fixed income depending on the momentum of the equities measured each month. We will start by using 252 days (1 year) as a period to measure momentum.
Operating a single entry point generates scattered results. We could say that the problem is that the system is not robust. It would have to work the same or a more similar way even if we varied the points of entry. Many of these systems are designed, and tested, with end-of-month data. However, diversification does not end there.
When we start designing an investment model, we look for and analyze many combinations of parameters in order to exploit an advantage. These combinations are filtered until you find the winners. And finally, the robust zone, which is the one that contains a series of stable combinations over time. It is usual to take the best combination of the robust area and operate it.
However, these parameters are sensitive to variations. What has historically been the best combination may not be so in the future. That the advantage exists and that it is robust, within a zone of parameters, is necessary to operate the model. But the best combination within the robust zone may not always be.
Two combinations of parameters can lead to small differences in the operation (an operation that you don’t take, an output that you do afterward, etc.) but that in the long term materializes in large differences in the results. This sensitivity grows as the uncorrelated parameters of a system increase. It is what is known as “specification risk”.
This sensitivity is closely linked to the types of systems. A permanent portfolio has very little sensitivity, while trend systems, where binary decisions are made (e.g. you are either 100% in equities, or you are 100% in fixed income), the sensitivity is very high. This also applies to intraday trading systems, factor-based investment systems, and all those using parameters. The difference between them shall be the sensitivity of the results to slight variations between the parameters.
This sensitivity may pose a risk of obtaining fewer results than expected by the model. When we started designing our system, we decided to use 1 year as a period to measure momentum and rebalance every 4 weeks. These two parameters appear to be very “current”, but simply that simple decision can lead to very disparate results than would have been obtained with other parameters close to each other.
In the following example we can see how using the variations for the same system, but using 3-4 weeks of rebalancing or 10-11-12 months of rebalancement, the signals begin to decouple gradually. This does not seem to matter, but in the long run, it can lead to large differences. And the more complex the system and the more variations the parameters allow the more they can be enlarged. If instead of an average, it were taking 2 or varied between simple or exponential, the differences in the signals would become more and more frequent.
Specification Risk In the Long Term?
Let’s look at this from a longer-term perspective. As the objective of the article is to raise awareness of the sensitivity to the parameters of some systems, we will continue working with a very simple model. The model will evaluate the momentum at 10-11-12 months of SPY, ignoring the last or 2 months. If positive, purchase SPY; if negative, TLT. Every 3-4 weeks it evaluates the signal and takes positions.
Therefore we have 3 parameters that vary very slightly. The period in which we measure momentum (3 options), the number of recent months we ignore (2 options), and each time we rebalance (2 options). 12 combinations in total. The reason we ignore months when measuring momentum is that assets have different long-term and short-term behaviors. Short-term equity can have an average reversal effect that can affect the long-term trend. Between two systems that differ in that one rebalances every 3 weeks, and another every 4, we find an impressive difference.
We have to know that none has been especially better throughout history. This means that making a decision today about which system is going to work in the future is taking the risk of choosing the worst of the 12. If we do, we will clearly decrease the overall profitability.
There are no longer only differences in profitability, but also in maximum losses. That small difference in signals, at certain times in the market, produces devastating effects. It can leave a system permanently behind. And the truth is, this choice has a big component of chance. It could not have been known in advance what combination, of parameters, would have been appropriate.
This system is designed primarily for educational principles. But it has the basic features of trend systems used by industry and many private investors. If we had decided to apply only the winning combination of parameters, between 1999 and 2009, under the pretext that it was clearly the best combination, we would have found the losing combination between 2009 and 2019. What would have been our mistake? Was the system not robust? The advantage of trend systems is there, but sensitivity to parameters is usually overlooked. The error is not the choice of that particular set of parameters, the error is to choose only a set and not to diversify.
When designing these systems and realizing that the sensitivity of the parameters is high, the first measure is usually to increase the rebalance frequency. It is a natural instinct, but not only is it not beneficial but it is highly harmful. You will find models with many more operations (currently these models rebalance between 20 and 25 times in 20 years) have the same sensitivity to the other parameters. They would still not be diversified. Like the proposed solution to avoid “Timing Luck”, the solution would be to operate all systems, creating an assembled joint system.
Operating the whole of the systems guarantees us to obtain the really expected returns of the trend system, eliminating the risk of choosing the worst of all. In addition, the low rate of rebalancing, of this type of system, plus the fact that much of it is rebalanced on the same days, does not make the costs much higher. For other types of systems, a balance would have to be found between the costs of increased operations and the benefits of risk diversification. This also has an extra benefit: by assembling uncorrelated systems at certain times (when some are long from SPY and others from TLT), while profitability will be the average. In addition, volatility will be lower, producing a better return-volatility ratio.
Specification Risk: Conclusions
This study will demonstrate that operating a single combination of parameters, of a model, is risky even though it has been the best combination in the past. Just as diversification between assets is important and, as we saw in previous articles, the entry points also affect the results of the operation, the diversification between sets of parameters, of a model, is also necessary to reduce risks.
“The purpose of this study is to show that operating a single combination of parameters, of a model, is risky even though it has been the best combination in the past.”
In the trend system used this sensitivity, between values, is very high since the decay, which can occur from one week between systems, causes that in the long term the results differ markedly. However, this point is applicable to the vast majority of investment models. Even two identical systems that in the past have not had any difference in the signal, having different parameters, can produce different results in the future without knowing a priori which would have been the best combination.