A historical simulation may seem like a simple process to perform; however, it goes beyond creating trading rules and introducing them into the simulation software. Within the software itself, some limitations can diminish the accuracy of the results and, in this way, overestimate or underestimate the possible results that the strategy would achieve in the real market.
This educational article explores the importance of precision in the simulation process and some simulation software problems.
Importance of the Accuracy
The developing process of a trading strategy that systematically creates trades can be tested on a historical simulation procedure with relative confidence and accuracy. However, like all software, it can be affected by precision and lead to errors, which can increase, especially when looking for a historical simulation.
In this regard, the developer should not rule out a possible error in the simulation software, nor can it leave to chance the computer simulation’s lack of precision, as this may drag unpleasant results to the investor. In consequence, the developer should address the software errors in the best possible way.
The developer must seek to make the simulation as realistic as possible to address the accuracy problem. To achieve the desired accuracy, the historical data exchange must be as close as possible to real-time executions.
Thus, according to Robert Pardo (2008), achieving greater accuracy in the historical simulation will require two things: understanding software limitations and using conservative assumptions about costs and slippage in their various forms.
Ignorance of historical simulation software limitations can lead the developer to a false confidence sentiment or be overly pessimistic about the historical simulation results. The most common constraints are:
Rounding of Data
The absence of the actual market price for the use of data rounding may have a cumulative effect on market entry and exit orders, which could lead to a cumulative effect, both positive and negative, on the trading strategy’s performance.
The problem of rounding in the historical simulation may lead to recording orders that, on a real market, would not have been executed or orders that would have been executed in real-time but not recorded by the simulation. A second error is a recurring understatement or overstatement in the strategy’s profits due to rounding.
The developer must determine if the historical simulation results are consistent with its real market results.
Finally, rounding errors will significantly impact a strategy that seeks small profits per trade. On the contrary, the impact will be less on those strategies that are slower or longer-term.
Price on Limit Orders
This error is a recurring problem in simulation software, and, in particular, it is presented in counter-trending strategies, which use pending orders to enter the market. In other words, the countertrend strategy places a buy limit when the market is falling and a sell limit when it is developing a rally. Contrary to a stop order, the execution of the limit order is not guaranteed.
The problem arises when the developer performs the historical simulation, and the software assumes that all limit orders will be executed during the simulation. However, according to Perry Kaufman (1995), up to 30% of all limit orders are not filled.
Faced with this problem, the strategy developer must define a security level to ensure that the order will be filled, increasing the probability of execution in the real market. This additional rule might consider that the price penetrates an additional distance to consider the order as executed.
Finally, this rule will not necessarily ensure that all limit orders will be executed; however, it will produce a more realistic approach to the simulation process.
The historical simulation process may present some problems generated by the code that the developer should be aware of. These inaccuracies can create false confidence by overestimating the results or drive it to be too pessimistic due to underperformance. These problems are mainly price rounding and limit-order executions.
In the first case, price rounding may induce the simulation software to execute input or output orders at levels other than those filled in the real market. To overcome this limitation, the developer must verify whether the strategy’s results are the same as the strategy would obtain in the real market.
The second error arises from counter-trend systems making use of limit orders. In this case, the developer must consider that not all limit orders are filled in the real market. Thus, the simulation could lead to overestimating the strategy’s results, creating a false optimism of the obtained performance. To mitigate this problem, the developer could introduce an additional requirement of an extra distance that the price should penetrate for the limit order to be executed.
Finally, in view that historical simulation software has limitations, the strategy developer should verify whether the simulation results are similar to those obtained in the real market.
- Pardo, R.; The Evaluation and Optimization of Trading Strategies; John Wiley & Sons; 2nd Edition (2008).
- Kaufman, P.J.; Smarter Trading – Improving Performance in Changing Markets; McGraw Hill; 1st Edition (1995).