Tag: Walk-Forward Analysis

Categories
Forex System Design

Introduction to Walk-Forward Analysis – Part 2

Introduction

Introduction

Introduction

In the previous part, we introduced the walk-forward analysis concept, its objectives, and its advantages. This educational article will continue discovering the benefits of using the WFA and how to set it up.

Walk-Forward Analysis and Market Impacts

The walk-forward analysis provides information about the impact of changes in trends, volatility, and market liquidity on the performance of the trading strategy or system. Generally,  when these changes occur, they arrive at a fast pace, heavily degrading the trading performance.

The WFA may extend its study in a wide range of time; however, analyses and evaluates the trading performance by separate windows. The broad range of results obtained by the study could provide the developer with a piece of useful information about the market changes impact the trading strategy performance.

Parameters Selection

As a robust optimization system, the WFA can provide the most appropriate parameters for real-time trading.

Simultaneously, the walk-forward analysis provides the strategy developer with the duration of the optimal period of time in which the set of parameters will consistently produce real-time benefits before the deterioration in trading system performance occurs.

Statistical Rigor in the Walk-Forward Analysis

As mentioned above, a large amount of data provides greater confidence in any phenomenon’s statistical study. This concept is also valid in the walk-forward analysis.

In this context, the walk-forward analysis must be large enough to produce several trades such that a large amount of data can be generated for the study. According to Pardo, in his work, he says that a WFA must be as long as possible, usually at least 10 to 20 years, whenever possible. Finally, he adds that these multiple walk-forwards combined performance will often be sufficient to produce the statistical rigor required in the analysis.

As a result, WFA’s multiple optimization windows will be able to give the developer a better idea of how the trading strategy will behave in the face of market changes.

Developing the Walk-Forward Analysis

A walk-forward analysis consists of two stages. In the first section, traditional backtest optimization is developed. The parameters of the trading strategy are analyzed using a sample established according to the developer’s objectives.

The second stage, which is the one that characterizes a walk-forward analysis, evaluates the performance of the parameters using an additional sample that was not used during the previous optimization stage.

The walk-forward analysis process requires the following elements to be set:

  1. Scan range for variables to optimize. The developer must define the time frame in which the trading strategy’s optimization should be performed. The developer should consider that the scan range uses a computational resources level that it will use to evaluate and weight the parameter to be optimized. In this regard, an exploration in a small number of historical simulations will consume less computational resources than a more extensive optimization.
  2. Identify a target or a search function. The developer must define what the purpose of the optimization study is.  A usual target can be a mix of the normalized average trade return (the reward/risk factor), the standard deviation of this figure, and the percentage of winning trades. These three factors will define the quality of any trading system. A fourth key factor is the number of monthly trades delivered by the system.
  3. Size of the optimization window. Generally, this optimization range can vary from 3 to 6 years. This duration depends on different factors such as, for example, the market, the type of trading strategy, the confidence level required by the developer in the optimization results, among other factors determined by the developer.
  4. Size of the walk-forward out-of-sample window. This period is defined based on the optimization window. In most cases, this window can be between 25% and 35% of the optimization time.

The length of the optimization window is determined by:

  1. Availability of data. Depending on the type of market and accessibility of the data to perform the analysis, the developer might find a restriction on the amount of historical data needed to perform strategy optimization.
  2. Trading strategy style. A short-term trading strategy should require a smaller optimization window than a long-term strategy.
  3. The pace of trading strategy. The pace of strategy is highly variable and tends to vary from strategy to strategy. For example, a long-term strategy, such as swing trading, will slower than a short-term trading strategy, which will require less time to produce the same number of trades.
  4. The relevance of data. The relevant data depends on a large number of factors that could empirically be determined. However, Pardo proposes a basic guideline stating that a short-term strategy can be tested using one or two years of data, whereas an intermediate-term strategy would require two to four years, and a long-term strategy four to eight years of data.
  5. The validity of the strategy’s parameters. The developer expects the parameters employed will generate benefits in the historical simulation. Furthermore, it also requires a trading strategy to produce profits in real-time market trading.

Finally, the walk-forward analysis is a post-optimization method, highly effective and revealing when it comes to discriminating a trading system’s robustness. Regarding that, the market is dynamic and changes periodically; the trading system should be re-optimized with a certain periodicity.

For example, a properly optimized short-term strategy could be re-optimized between three and six months of real-time trading. While the long-term between one and two years.

Conclusions

In this second and final part, we reviewed the basics of walk-forward analysis, characterized by providing the trading system developer with a powerful tool for testing, validation, and measuring trading strategy. Among the benefits that this stage of the development of a trading system provides we can mention:

  1. Measurement of robustness.
  2. Reduction of overfitting.
  3. Assessment of market changes in the trading strategy.
  4. Selection of optimal parameters for the strategy.
  5. Statistical reliability, when correctly applied.

Finally, despite the benefits provided by this analysis methodology, the strategy developer should consider that the trading system may need to be re-optimized with a certain periodicity.

Suggested Readings

  • Jaekle, U., Tomasini, E.; Trading Systems: A New Approach to System Development and Portfolio Optimisation; Harriman House Ltd.; 1st Edition (2009).
  • Pardo, R.; The Evaluation and Optimization of Trading Strategies; John Wiley & Sons; 2nd Edition (2008).

 

Categories
Forex System Design

Introduction to Walk-Forward Analysis – Part 1

Introduction

Introduction

Introduction

Once completed the optimization process of a trading system, the developer could develop an advanced strategy optimization method, the walk-forward analysis. In this educational article, we will introduce the basic concepts of this methodology.

What is Walk-Forward Analysis?

The Walk-Forward Analysis (WFA) is an advanced method for testing, validating, and optimizing a trading strategy, by measuring its performance using out-of-sample results. The WFA attempts to achieve the three objectives identified as follows.

  1. Determine if the trading strategy continues being profitable using unseen or out-of-sample price history.
  2. Find out the optimal values of the parameters to be used in real-time trading.
  3. Delimit the optimization window size and the period at which the system should be reoptimized.

Measuring the Robustness of a Trading System

A robust trading system is profitable, even when the actual market conditions change. These profits tend to be consistent, with the results obtained in the historical simulation stage. In other words, a walk-forward analysis allows the system’s developer to determine if the trading strategy can produce real-time profits.

The system’s developer is able to compute the robustness of a trading strategy using a statistical criterion called Walk-Forward Efficiency (WFE) ratio, which measures the current optimization process’s quality.

The WFE ratio is computed by the out-of-sample annualized rates of return divided by the in-sample returns. A robust trading strategy should achieve reliable performance, both in-sample and out-of-sample data. A 100% ratio would indicate that the out-of-sample figures match exactly the returns obtained with the backtests. That would indicate the system is sound and reliable.  Pardo, in his book The Evaluation and Optimization of Trading Strategies, comments that “robust trading strategies have WFEs greater than 50 or 60 percent and in the case of extremely robust strategies, even higher.”  A WFE ratio below 50% would indicate poor performance. Finally, ratios over 100% are suspicious, and the system should be analyzed to discover the reason for this behavior.

WFA and Overfitting

Overfitting is a condition resulting from an excessive readjustment of the strategy’s parameters aimed at artificially improve its performance. An overfitted strategy is likely to lose money in real-time trading, even when a historical simulation would present terrific gains.

By its nature, a walk-forward analysis is a cure for overfitting abuse. This edge comes from the fact that the WFA evaluates the strategy’s performance using data not belonging to the optimization process.

By using WFA, the developer is certain about the robustness of a trading strategy. In other words, a not too robust trading strategy would not pass a walk-forward analysis.

Considering that a large number of samples increase the statistical validity and confidence, it is unlikely that an unprofitable trading strategy would make significant profits using a large number of samples. Consequently, a poor trading strategy delivering gains could be the product of chance.

Conclusions

The Walk-Forward Analysis (WFA) is a powerful tool for testing, validating, and measure the quality of a trading strategy, which is characterized by the use of out-of-sample data to evaluate its performance. In this educational article, we have presented the advantages of using WFA to evaluate the strategy’s robustness and how WFA can help the developer avoid overfitting, increasing confidence through an objective statistical measure.

In the next educational post, we will continue reviewing the benefits of using the walk-forward analysis and setting up the walk-forward analysis.

Suggested Readings

  • Jaekle, U., Tomasini, E.; Trading Systems: A New Approach to System Development and Portfolio Optimisation; Harriman House Ltd.; 1st Edition (2009).
  • Pardo, R.; The Evaluation and Optimization of Trading Strategies; John Wiley & Sons; 2nd Edition (2008).