Once we develop an idea into a system, it seems straightforward to test it fully assembled, with entries, filters stops and exits. The problem with this approach is that one part of the system may interfere with the behaviour of another part, and its combination may result in such bad results that we could conclude that the original idea has no value at all.
On the positive side, by testing every element of the system independently we’d be able to assess its own characteristics much better, strengths, and weaknesses, and we’d be qualified to better correct and improve it.
For these reasons, it’s much better to create methods to test portions of a trading system in isolation. That cannot be perfect though because a system is an interconnection of its parts, but this way leads to a much better end result.
One of the most revealing situations a believer in Technical Analysis may experience is how their assumptions, when back-tested using a proper method, are shattered by a reality check. Well-known studies that many people are using for trading produce mediocre results when faced with a rigorous test on a mechanical entry system.
According to many authors, all you can ask for of an entry is to give you a bit better than random potential. Once you get that, it’s the exit strategy that is the one in charge to capture as much profit as possible.
One key statistics of an entry signal is the percentage of winners. Everything else being equal, it is preferable to have a high winning system to a lower one. But we must remember that the profit equation includes the reward to risk ratio as well, and this aspect allows having good profitable systems with percentage winners below 50% if the reward to risk ratio is higher than one.
Day traders face a more difficult task. They must develop trading systems with percentage winners greater than 50%, or devise forms to make their profit to loss ratio greater than one, it is also complicated to attain this in short time frames.
On long-term and short-term trading systems, if we wanted to get high percentage winners we’d need a highly precise entry signal; and while exits can be designed to take an optimum part of the generated profits on a trade, its task is much easier if the entry signal is a good predictor of the future price movement.
Methodology of entry testing
There are three methods to assess the quality of an entry signal.
- Fixed bar exit
- Fixed target and stop
- Random exit
Fixed bar exit
This is the most common approach. It’s also an excellent way to evaluate the time characteristics of an entry signal and observe the length of the price move. The idea is to test this using more than one setup. For example, you might be interested to know if a signal has predictive value at 3, 5, 10, 15, 20, and 30 bars. If it has predictive value at 15 to 30 bars, but this is missing at 3 and 5 bars, maybe your signal triggers too early. If it has predictive value in the shorter times but loses it on longer ones, perhaps the signal is too late, or the price movement ends before reaching those bars.
If the entry signal is sound, it should get into the market in the right direction with a winning success significantly higher than 50%. As a rule of thumb, it should show more than 55% winners over a range of markets or currency pairs. That is important, because, after adding stops this figure will decrease substantially, and the better this value is, the tighter the stop can be.
Fixed target and stop
If we set the stop and target at the same pip amount, we’ll get around 50% success if we use a random entry. Assuming the tested signal is better than random it’ll show a better figure. Therefore, we must just set the target and stop levels appropriate for the instrument we’re testing.
The concept of a random exit is to eliminate the impact on exits over entries, just seeing the ability of entries to generate profits. If the entry is almost always profitable, even with a random exit, then there is a very good chance that the entry has an edge.
Percentage of winners: A valid way to test the signal. As we’ve said, if we get significantly more than 50%, then our signal may have an entry edge, especially if its winning percent goes beyond 60 percent.
Average profit: We need not only winners but results. Although we are still dealing with just an entry signal, good results are a very good metric to judge its value. That avoids rejecting perfectly valid trading strategies, such as trend following, which typically show less than 40 percent winning rates but with high reward to risk ratios.
One way to assess the robustness of the entry is to perform an optimisation procedure and evaluate the percentage of variations that show good results. If you only get a handful of cases with positive results, the entry method is not reliable. If more than 70% show profits, then you got a successful entry signal.
Study example: MA crossovers using fixed bar exit
Below is a moving average (MA) crossover entry study. We are using a 20-bar exit. We observe that only 9 out of 52 crossover variations are profitable on a 5-year EUR/USD study, and if we observe the MA combinations that are profitable (not shown in the fig) this happens when the fast MA is longer than the slow MA; thus profit comes from fading the original idea.
So, let’s do the opposite study, by shorting when the fast MA crosses over the slow MA instead and buying when crossing under. In fig 2, we observe that the profitability of the inverted MA crossover is much more robust and reliable than the supposedly “good” forward signal. This shows that in intraday trading we cannot take anything for granted and that MA crossovers no longer work in the standard way.
As a final exercise, let’s choose a stop and target for this entry signal and look what comes out. Fig 3 shows a 3D map of the profitability of the combinations. Let’s choose the hill at MAF = 60 and MAS =25, a region that seems to show a robust behaviour.
As we see in Fig 4, this entry system is quite robust along the combinations of trail stop and target. Fig 5 also shows that the trail stop level isn’t critical, but a level at 0.1% will take us out of significant drawdowns and a very good win to loss ratio, at the expense of a fewer profits and 41% percent gainers.
A trailing stop at 0.3% will triple our drawdown with only a minor increase in profitability and a percent profit increase to 45%, that in my opinion isn’t worth the while.
On the target side, we observe that the hill of profitability is reached at about 0.6 % to 0.7%. Overall this system shows a ratio of profits to max drawdown above 5:1 when using a trailing stop of 0.1%. Below the equity curve using a 0.1% stop and 0.6% target.
By following the steps to evaluate an entry in an orderly manner, we started from a raw idea that showed wrong results and arrived at a possible trading system by tweaking the original idea.
We have just ended the preliminary study. Of course, further tests should be necessary and could possibly improve on our exit management, before committing any money to it.
The idea of this exercise was, of course, to use a trading platform and perform the necessary steps to test an idea, and then either discard it or transform it into a profitable signal, by examining what comes out of a rough process of optimisation, evaluating the resulting data as a whole.
Further readings from this series:
Building Winning Algorithmic Trading Systems, Kevin J. Davey
Computer Analysis of the Futures Markets, Charles LeBeau, George Lucas
Images were taken with permission from Multicharts 11, Trading Platform