Before you start you may wonder why a consistent trading strategy and what exactly this means. One of the biggest problems you will know when trading is that you don’t know what will happen tomorrow. We have a lot of data on what has happened so far, but every day the markets behave differently.
So you’re in need of a plan that can be adapted to a changing scenario. That’s why doing things right requires a strategy. A strategy that is able to be consistent when the market changes. Careful, I don’t mean by this that I will always win no matter what happens in the market. By consistent, I mean exploiting the inefficiency of the market for which it has been created and being successful over time (not always).
Importance of Backtest Trading
As I was saying, what you have in your hand is past data to measure how good your strategies are. We do this through what’s called a backtest, which is basically about extracting statistics from our strategies and evaluating how they’ve behaved.
The Big Backtesting Error: Over-Optimisation
The problem is when we use that data to create and measure strategies at once. This leads to over-optimization. To understand it better I will give you an example. Imagine that I tell you that if I create a winning strategy I give you a million euros but you don’t have much idea of trading. Surely you take a couple of pointers and see how they’ve worked and said the typical, “If I had come in here and gone out there…” and present me with a strategy on incredible paper. A strategy that during that time seems infallible. And that is that during that time frame surely will be.
Quite possibly if I apply this strategy I won’t get even close to the results I could expect from backtesting. Why? Basically, because I have been playing with the marked cards. I have looked for a good result without considering the strategy. I’ll explain it to you.
How to Perform a Good Backtest
Then you will surely wonder if it is possible and how we can measure the performance of our trading system without falling into this bias. The answer is to divide the time frame or the data.
Let’s go with another example to understand it better. We are doing a backtest of our strategy from 2013 to 2019. What we’re going to do is use 70% of our data to build the strategy and 30% to measure its performance. This is something like seeing what would have happened if I had actually implemented this strategy in that 30% of the time horizon. This is actually what we’re interested in, to see how it would have worked.
In Sample (IS): 70% of the data in the example above is what we call in a sample or inside the sample. I remind you that’s what we use to build our system.
Out of Sample (OOS): On the other hand 30% is what is known as out of sample or out of sample. The data we will use to measure your performance.
The Importance of Backtest
We might think that the really important thing when we observe a backtest is that the off-sample period presents favorable statistics. But on top of this, the ideal is for periods in sample and out of sample to have a similar behavior.
This tells us that the strategy behaves in both periods in the same way and shows us a clear sign of robustness. Look at our case as we can see that in both, the performance is very similar. We are facing a strategy that is a candidate to move to real.
Advanced Techniques
There are more advanced techniques to evaluate trading systems in this way, one of the best known is the walk forward test. In a few words it is how to make this division in the data but in different stretches in time. It is a question of doing this division (in sample and out of sample) in a segmented way to go a step further.
What to Do Now
My advice is yes or yes to measure the performance of your strategies with a backtest and to do it with an off-sample period. Well 20%, 30% or 40% but do it. Thus you will avoid biases and errors that make us see very beautiful profit curves that then are not real. And this means avoiding wasting money, time and frustration. This is just one more step towards objectivity.