Metatrader 4 Backtesting, Reliable or Not ?
As many of you may already know, the Metatrader 4 trading platform has a strategy tester whose objective is to evaluate a strategy based on historical data, rather than executing an expert advisor for an extended period of time. But the question for many remains, is it reliable ?
This question is of the highest importance as many decisions could be based on this feature of the program. Someone who believes in the backtester being accurate will have no problem in using it as a filter for his expert advisors and also using it as a measure of an unknown strategy.
First of all, we have to be very clear. There is only one way to test the accuracy of the strategy tester in Metratrader 4. First, we need to run an expert advisor in a demo account for a given period of time, second, we must backtest this exact same expert in that exact same period of time. The comparison between trades taken in both settings will let us know if the backtester is or is NOT accurate.
Many people say backtesting is not accurate because the expert was profitable for two months of backtesting, then wiped out the accound in real life. This is actually NOT an indication of a flawed tester (the expert could have behaved this way naturally because of changing market conditions). To be clearer, an expert being profitable in backtesting and not in forward testing is a necessary but not sufficient condition to declare the backtester unreliable. To be sure, you must perform the test suggested on the third paragraph.
To do this, you must first set the backtester so that you can get the best possible results. For this I have set my metatrader platform with alpari historical one minute data dating back to 2004. You also want to use the every tick mode and use every less time frame in your analysis. I have done small tests for different expert advisors and I have arrived to the following conclusions.
- Expert advisors that open and close trades in a 1-4 minutes timeframe CANNOT be accurately or even approximately tested with the Metatrader 4 backtester, this, because the tester uses fractal interpolation. It guesses what happens under the one minute candlesticks.
- Expert advisor behavior is not predicted in any case when a sudden move occurs in a one minute candlestick. When an expert trades a 100 pip move which occured in under a minute (like after some NFP releases). In many cases the results in reality and with the tester are fairly different.
- Experts that trade 1-3 times a day and have wide (50 or more) stops and take profit levels are usually predicted far more accurately than experts with tighter levels.
- Experts that trade on the closing price of bars sometimes exhibit abnormal profits because of "divination" of the strategy tester. It uses future knowledge by mistake.
- To deem an expert advisor profitable by backtesting, at least a 3 year analysis are necessary. This supplies a wide range of market conditions, which would at least justify a demo testing of the expert advisor.
So is backtesting useless ? This is actually NOT the case. Would I buy an expert advisor based on backtesting ? Absolutely NOT ! The reliability of the results is very dependable upon the nature of the expert advisor and only access to the source code and understanding of the trading method can guarantee a critical grasp of the tester's results.
So personally I will continue to use metatrader's backesting as long as I know that the expert does comply with the requirements necessary for reliable testing. I will, however, never buy any commercial experts based solely on backtesting as it is - quiet frankly - easily fooled by programming manipulation.
This question is of the highest importance as many decisions could be based on this feature of the program. Someone who believes in the backtester being accurate will have no problem in using it as a filter for his expert advisors and also using it as a measure of an unknown strategy.
First of all, we have to be very clear. There is only one way to test the accuracy of the strategy tester in Metratrader 4. First, we need to run an expert advisor in a demo account for a given period of time, second, we must backtest this exact same expert in that exact same period of time. The comparison between trades taken in both settings will let us know if the backtester is or is NOT accurate.
Many people say backtesting is not accurate because the expert was profitable for two months of backtesting, then wiped out the accound in real life. This is actually NOT an indication of a flawed tester (the expert could have behaved this way naturally because of changing market conditions). To be clearer, an expert being profitable in backtesting and not in forward testing is a necessary but not sufficient condition to declare the backtester unreliable. To be sure, you must perform the test suggested on the third paragraph.
To do this, you must first set the backtester so that you can get the best possible results. For this I have set my metatrader platform with alpari historical one minute data dating back to 2004. You also want to use the every tick mode and use every less time frame in your analysis. I have done small tests for different expert advisors and I have arrived to the following conclusions.
- Expert advisors that open and close trades in a 1-4 minutes timeframe CANNOT be accurately or even approximately tested with the Metatrader 4 backtester, this, because the tester uses fractal interpolation. It guesses what happens under the one minute candlesticks.
- Expert advisor behavior is not predicted in any case when a sudden move occurs in a one minute candlestick. When an expert trades a 100 pip move which occured in under a minute (like after some NFP releases). In many cases the results in reality and with the tester are fairly different.
- Experts that trade 1-3 times a day and have wide (50 or more) stops and take profit levels are usually predicted far more accurately than experts with tighter levels.
- Experts that trade on the closing price of bars sometimes exhibit abnormal profits because of "divination" of the strategy tester. It uses future knowledge by mistake.
- To deem an expert advisor profitable by backtesting, at least a 3 year analysis are necessary. This supplies a wide range of market conditions, which would at least justify a demo testing of the expert advisor.
So is backtesting useless ? This is actually NOT the case. Would I buy an expert advisor based on backtesting ? Absolutely NOT ! The reliability of the results is very dependable upon the nature of the expert advisor and only access to the source code and understanding of the trading method can guarantee a critical grasp of the tester's results.
So personally I will continue to use metatrader's backesting as long as I know that the expert does comply with the requirements necessary for reliable testing. I will, however, never buy any commercial experts based solely on backtesting as it is - quiet frankly - easily fooled by programming manipulation.
Comments
Post a Comment