Introduction
The first issue of this series on trading systems finishes with a chart flow as in Fig 1, that sketches the steps needed for a good design of a trading system.
There are other ways around, of course, for example, from limited testing going directly to paper trading or small-sized trading, but this larger flow makes sure that any system that is profitable at the end of this pipe will be performing close to what’s expected.
Moreover, the designer and trader will be much more confident trading it and will be aware of what to expect in terms of returns, percent gainers, and drawdowns.
Of course, this a long process, taking months to a year, or even more to accomplish. That’s not an easy way of doing it, but, as Kevin Davey says “That’s how it’s supposed to be. Think about it for a second – if it were easy to find a strategy don’t you think others would have already found it and exploited it?”
Kevin Davey says that for him, strategy development is like a factory, a pipeline of ideas that get developed along the refining process until its output, as garbage or as a gold nut. Therefore, we need to keep our factory running all the time, filling it with new preliminary ideas.
That said, we have to build that factory, first. Throughout this article, we’ll deal with the factory building problem, therefore, we’re going to explore Fig 1 chart flow and find out what’s needed on each stage of the pipeline.
Trading ideas
Kevin Davey on his book(see reference below) gives a very good list of things to consider about idea gathering:
- Keep an updated list of ideas. Whenever you jump on a trading idea or something that intrigues you, write it down on your list
- Look for ideas: Ideas are around us in books, web pages, internet forums, and magazines.
- Dumb ideas are those never tested. Absolutely everything might be a good idea. Even the craziest idea might end as a good system.
- If you make a mistake while coding (or in your script if you aren’t coding), test it anyway. Penicillin discovery was accidental.
- If your idea results in a bad system, try the opposite: switch buy and sell signals and see what happens. Although it doesn’t always work, it might.
- Plan to test one to five strategies per week. With this amount of inputs, it may take six months, but you’ll end up with a pack of good trading systems.
- Find other traders and offer to swap ideas and strategies with them. Take what others have and build strategies around those ideas.
Limited testing
Goal setting
Before going on into either a limited or an extended testing, we need to define the specific objectives our system must accomplish.
A list of the items might be:
Likes:
- The possible markets the system is going to trade
- The Time-frame or time-frames applicable
- Minimum volatility to trade
- Allowed time intervals
- Purely automatic, Automatic entries and manual on exits or vice-versa.
- Reward-to-risk desired interval (aiming for 1.5:1 at least is a good starting point)
- Minimum percent of gainers we need to be comfortable with.
- Minimum quality criteria for the system:
- maximum allowed coefficient of variation
- Minimum quality metrics (Sharpe, Sortino, SQN …)
- Maximum drawdown length
Dislikes:
- Trading on strength / Trading on weakness
- More than X trades a day / less than X trades a day
- More than 5 losers in a row
- Targets too large / short
The idea
We take an idea from our list of ideas and we decide about how to develop the following details:
- Testing and trading platform
- Data needed
- market direction
- timeframe or bar size
- trading intervals (optional)
- Entry rules
- Allow long and allow short signals
- trigger long and short signals
- Exit rules
- Long and short stops
- trail stops
- volatility stops
- Profit targets
- Algorithm programming of decision chart flow in manual execution
Testing, optimizer, and trading platform
Developing a system by hand is possible, but it takes a lot of effort, and at least, it needs some kind of programming to perform the Monte Carlo testing, and fine-tune the position size to our particular needs and tastes.
Metatrader
Currently, the most popular system for the forex markets is Metatrader. It uses a C++ variant called MetaQuotes Language MQL. The latest version is MT-5, although MT-4 is still used by many forex brokers.
The following are the key features of the MQL-5 language:
- C++ syntax
- Operating speed close to that achievable with C++
- Wide range of built-in features for creating technical indicators
- Open-Cl support for fast parallel executions for optimization tasks without the need to write parallel code.
- Wide variety of free code for indicators and strategies with a strong mlq4 and mlq5 community.
Python
Python is a terrific high-level programming language with tons of features and a huge number of libraries for anything you can imagine, from database to scientific, data science, statistics, and machine learning, from logistic regression to neural networks.
Python has a specific library to backtest and optimize MT4 libraries and expert advisors.
Python-Metatrader can also be bridged to Meta Trader using ZeroMQ free software distributed messaging. (http://zeromq.org/intro:read-the-manual)
https://www.youtube.com/watch?v=GGOajzvl860
Python enables the implementation of different kinds of strategies, compared to those developed by typical technical analysis, although there are technical analysis libraries in Python. As an example, Python makes it easy to develop statistically-based strategies, for example, stats-based pivot points.
The implementation of Monte Carlo permutation routines takes less than 4 lines of code, And Python has a lot of machine learning functionality, including parallel GPU-enabled neural network libraries, such as TensorFlow, a terrific deep learning neural platform by Google. https://www.tensorflow.org
There exists a large community of quants developing stats-based strategies in Python, and lots of free code to start from, so ZeroMQ is a serious alternative to direct design in MQL.
One of the most popular Python packages is the Anaconda distribution. Anaconda incorporates Jupyter Notebooks, an interactive Python web-based environment that allows development of Python apps as if it were a notebook (Fig. 3). Anaconda includes, also, an integrated software development IDE called Spyder (fig. 4)
Anaconda distribution may be found at https://www.anaconda.com/download/
Data
Forex historical data bars can be acquired for free from the broker for major pairs crosses, and exotic pairs, with more than 5 years of one minute, via MT’s the historical data center (F2). Starting from the minute bars, other timeframes can be recreated easily
Tick and sub-minute data aren’t available for free, so we should consider if the idea needs data resolutions beyond the minute.
Forex traders are fortunate in that their data is already continuous through time. Futures trading needs to combine several contracts on a continuous contract because futures contracts expire every three months.
The problem is that the expiring contract prices are slightly different from the starting prices of the new contract because future prices are affected by interest rates since its value is computed taking into account the cost of the money, and this cost varies as the distance to its expiration approaches.
If you need to build your own continuous contract, there are several ways. The simplest is just to put them side by side and let the gap appear, but, obviously, this is wrong, so the second easier way is to back-adjust the price in the old contracts, by subtracting the distance in points from the current contract’s starting price to the ending price of the latest expiring contract. That’s ok, but you may end up with negative prices if you are doing that a lot of times. The other problem with this method is that you lose the actual historical price values and, even worse, the original percent variations.
The best method, in my opinion, is to convert prices to ratios using the formula:
Price change = Closei – closei-1 / closei-1
Then go back to the first price of the current contract and perform the conversion back from there on the historical data series.
Data Cleansing
Historical data are rarely perfect. Spurious prices are more common than most think. Also, it may be desirable to get rid of price spikes that, although correct, are so unusual that they should be ignored by our system. Therefore, it may be desirable to include a data cleansing routine that takes away unwanted large spikes.
A way to do that is to check each bar in the price history and mark as erroneous any bar if the ratio of its close to the prior close is less than a specified fraction, or greater than the specified fraction’s reciprocal. All marked erroneous dates won’t be used to compute any variable, indicator or target, or they can be filled with the mean values of the two neighboring bars.
Data normalization
The common way to design a trading system does not involve any price normalization or adjustment, besides what is needed to create a continuous contract in the futures markets.
There are two kinds of technical studies, those whose actual value at one bar is of key importance in and of itself, and those whose importance is based on their current value relative to recent values. As examples of the first category, we may consider PSAR, ATR, Moving averages, linear regression, and pivot points. Examples of the second category are stochastics, Williams %R, MACD, and RSI.
The reason to adjust the current value of an indicator to recent values is to force the maximum possible degree of stationarity on it. Stationarity is a very desirable statistical property that improves the predicting accuracy of a technical study.
There are two types of price adjustment: Centering and scaling. Centering subtracts the historical median from the indicator. Scaling divides the indicator by its interquartile range.
Centering
There are several ways to achieve centering. One of them is to apply a detrending filter to the historical price series. The other way is to subtract the median of some bar quantity, for example, 100 to 200 bars.
Scaling
Sometimes centering the variable may destroy important information, but, we may want to compensate for shifting volatility. It may happen that a volatility value that is “large” in one period might be “medium” or even “short” in another period, thus, we may want to divide the value of the variable by a measure of its recent value range. The interquartile range is an ideal measure of variation because it’s not affected by outliers as a classical standard deviation would be.
The formula to do that is:
Scaled_value = 100 * CDF [ 0.25 * X/(P75 – P25) ] -50
Where CDF is the standard normal Cumulative Density Function
X is the unscaled current value
P75 and P25 are, respectively, the 75 and 25 percentile of the historical values of the indicator.
Spectral Dilation
John Ehlers coined the term “spectral dilation” to signal the effect of the fractal nature of the markets and its profound effect on almost all technical indicators.
The solution he proposes to get rid of this effect is a roofing filter. A roofing filter is composed of a high pass filter that only lets frequency components whose periods are shorter than 48 bars pass in. The output of the roofing filter is passed through a smoother filter that passes components whose periods are longer than 10 bars.
Market direction
We must decide whether we are going to trade both directions indistinctly or if we should define a permission rule for every direction.
It seems straightforward the need to define a trend-following rule, but it’s not. Sometimes that rule doesn’t help to improve performance. On the contrary, it forbids perfectly valid entry signals and it hurts profits and other system metrics.
To assess the efficacy of a market trending signal, the right way is to test it after having tested the main entry signal. We should weight any possible improvement, but focused return on account, not merely in an increase in total profits.
Timeframe or bar size
Many traders don’t pay attention to this variable, although it plays an important role in the performance of a system. Timeframe length is really an important issue when developing a trading system because it will define a lot of things:
- The length of the timeframe is inversely proportional to the number of trade opportunities available on a given period.
- Length is proportional to the time it takes to close a trade.
- It’s also connected to the mean achievable profit. An hourly bar would offer a channel width much wider than a 5-minute bar. Since trading costs are a fixed amount for every trade (spread + commissions) timeframe length is proportional to the ratio Rewards/costs.
- It directly connects with risk. A longer timeframe needs longer stops, so we should lower our position size for the same monetary risk
The forex three basic categories are:
- Midterm: from 2-hour bars to a couple of days. It may be used with swing trading or similar techniques.
- Short term: From 15 minutes to 1-hour bars.
- Very short term: from seconds to 15 min-bars.
Time frames of popular strategies:
- Scalping: It’s a very short-term strategy. From seconds to a few minutes to complete a trade. It usually takes less than 5 bars to complete a trade.
- Day trading: from very short-term to short-term. From minutes, up to 1-hour bars, although the most popular are 5, 10 and 15 minute-bars. It takes from 3-5 minutes up to hours to complete a trade. Open trades are closed before traders stop their trading session.
- Range trading: This type of strategy doesn’t rely on a time frame but on a range breakout. So, its length depends on the range size. A small range takes less to be crossed through, while a large range may take 30 minutes or more. The most useful range will be that one that transition on average every 3-6 minutes.
It is not advisable to choose very short time frames. There, the market noise is very high and the timing for entries and exits is much more critical. The most critical parameter of a trading system is profitability with low variance. Profit objectives can be easily achieved using proper position size, but not over-trading in shorter time frames. Those very short time frames are only good for your broker.
General rules for entries
Once we have an entry rule we need translating it into a computer language. Computer languages are a formal and unequivocal description of a set of rules to do a task. If we need to test our idea in 5 years of one-minute data we surely will need an advanced trading platform such as MT5 or MT4 at least. But, even if we aren’t going to do that we still need to formalize our rules.
To do that a good solution is to implement our system in some pseudo code. This is, even, advisable as a first step before programming it to real code. Python users are very fortunate because Python code matches perfectly pseudo code. An example of pseudo code might be:
# Pseudo code for a simple 2-MA strategy using the slope instead of the crossover. Inputs: minRR = 1.5 PeriodLong = 15 PeriodSHort = 5 NN = 1 StopLong = MinLow(Low, 10) - 3 pips # it takes the low point of the last 10 bars - 3 pips StopShort = MaxHi(High,10) + 3 pips # it takes the max of the last 5 bat TargetLong = MaxHi(High, 20) -3 pips # Target is max of the last 20 bars – 3 pips TagetShort = MinLow(Low, 10) + 3 pips # Target is the min of the last 10 bars + 3 pips # Go long if both moving averages are pointing up if Long_avg[0] > Long_avg[1] and Short_avg[0] > Short_avg[1]: GoLong = True GoSort = False RR = (TargetLong –Price) / (Price-StopLong) # Go short if both moving averages are pointing down if Long_avg[0] < Long_avg[1] and Short_avg[0] < Short_avg[1]: GoLong = False GoSort = True RR = (Price - TargetShort) / (StopShort - Price) If Golong and RR>minRR: Buy NN contracts at Price, limit If Goshort and RR >minRR: Sell Short NN contracts at Price Limit # Stops and targets If myposition > 0: Sell at StopLong Sell at TargetLong If myposition < 0: Buy to cover at StopShort Buy at TargetShort
Bullet points for creating good entries:
- KISS principle applies: The “Keep it simple stupid” principle devised by the US Navy. If you can’t explain it in simple terms you’ll have a problem while converting it to rules or code.
- Limit the parameter number: The higher the number the higher the probability of overfitting. If you have just 3 entry parameters, then with exit and filter parameters you’ll end up with 8 to 10 that need to be optimized. Try to limit that to no more than two.
- Think differently. For example, MA crossovers have been used extensively. If you want to try them, think on how to innovate using them, for example, using MA’s against MA crossover users by fading the signal as a kind of scalping and look what you get.
- Use a single rule first, test it and continue adding another one and observe its effect in performance, so you’ll know if it works or is junk.
General comments on exits
Exits seem to be the poor relative in the trading family. Most people pay little attention to exits. They seem to believe that a good entry is all that matters. But I’ll tell you something: Exits can turn a lousy entry strategy into a decent or good system, and the contrary applies: it can turn a good entry strategy into a loser.
Stop loss define the risk: The distance from entry to stop-loss together with the size of our position are the variables needed to compute the monetary risk of a trade.
Profit target to entry define the reward, and with reward and risk, we can define our reward to risk ratio. This ratio and the average percent of winners is all we need to make a rational decision to pull or not to pull the trigger on a particular trade.
If for example, our system’s average percent winners are 50%, and the risk is two times bigger than the expected reward, it’s foolish to enter that trade. We’d require that the reward was at least a bit bigger than the risk to have an edge.
The usual exit methods are:
- Technical-based stops: Stops below supports on long entries and above resistance on short entries
- MAE Stops. Maximum adverse execution is a concept devised by John Sweeney. The main idea is: If our entries have an edge, then there will be a behavior on good trades different to bad trades. Then if we compute the sweet spot where the good trade almost never reaches, this is the right place to set our stop (In following articles, we will develop on this idea).
- Break-even stops: At some point when the trade is giving profits, many traders move the stop loss to break-even. This is psychologically appealing, but it may hurt profits. Moving the stops to break-even should be based on statistically sound price levels that, not on gut feeling, and should be based upon its goodness, not to make us more comfortable.
- Trail stops: As the trade develops in our favor, the stop-loss is raised/lowered to a new level. Trail stops may be linear or parabolic.
- Profit Targets. Profit targets really are not stops. Exits on targets are usually accomplished using limit orders. As with the MAE stops, we should test the best placement for profits statistically instead of fixed money targets.
©Forex.Academy
References:
Building Winning Algorithmic Trading Systems, Kevin J. Davey
Computer Analysis of the Futures Markets, Charles LeBeau, George Lucas
Statistically sound Machine Learning for Algorithmic Trading of Financial Instruments, Aronson and Masters