We will now repeat our experiment with the 900 trend trading strategies, but this time with trades filtered by the **Market Meanness Index**. In our first experiment we found many profitable strategies, some even with high profit factors, but none of them passed **White’s Reality Check**. So they all would probably fail in real trading in spite of their great results in the backtest. This time we hope that the MMI improves most systems by filtering out trades in non-trending market situations.

### 900 systems experiment revisited

I have been informed by readers that I committed two mistakes, or at least inaccuracies, in the previous experiment. First, I didn’t detrend the price data. Second, I used the equity curves instead of balance curves for determining the profit factor. I didn’t detrend the prices because the systems traded long/short in a symmetric way, and I supposed that this would eliminate any trend bias. But even if this was true back then, it is now not true anymore: filtering trades by MMI or other means can introduce asymmetry. Also, calculating the profit factor from the balance curve makes indeed more sense because you’re interested in the end profit of the trades, not in their interim behavior. Therefore and for the sake of comparable results I will now and in the future use detrended trade returns and balance curves for such experiments.

The original test, repeated with the modifications, produced a wider profit factor distribution due to eliminating intermediate returns. But the outcome of the experiment was the same. The statistic (including trade costs) did not change much, however the profit factor distribution (without trade costs) did. This is the new WRC histogram of the original 900 systems (best system vs. bootstrap-randomized returns of all systems):

Although the best system (black bar, a system using ALMA) is at the right side of the distribution, still 11% of random systems were better. The system does not pass the WRC at the required 95% confidence level. This turned out very different when filtering trades with the MMI.

### The MMI experiment

This is our script **TrendMMI.c** for the new experiment:

// helper function: remove systems that exceed the 4 months lookback periodint checkLookBack(int Period) { if(Period >= LookBack/TimeFrame) { StepNext = 0;// abort optimizationreturn LookBack/TimeFrame;// reduce the period} else return Period; }// calculate profit factor and remove systems with not enough tradesvar objective() { if(NumWinTotal < 30 || NumLossTotal < 30) { StepNext = 0;// abort optimizationreturn 0;// don't store this system} else return WinTotal/LossTotal;// Profit factor} var filter(var* Data,int Period); void run() { set(PARAMETERS|LOGFILE); Curves = "DailyBalance.bin"; StartDate = 2010; BarPeriod = 15; LookBack = 80*4*24;// ~ 4 monthsDetrend = TRADES;// detrend trade resultswhile(asset(loop("EUR/USD","SPX500","XAG/USD"))) while(algo(loop("MM15","MH1","MH4"))) { TimeFrame = 1; if(Algo == "MH1") TimeFrame = 1*4; else if(Algo == "MH4") TimeFrame = 4*4;// no trade costsSpread = Commission = RollLong = RollShort = Slippage = 0; int Periods[10] = { 10, 20, 50, 100, 200, 500, 1000, 2000, 5000, 10000 }; int Period = Periods[round(optimize(1,1,10,1),1)-1]; var *Price = series(price()); var *Smoothed = series(filter(Price,Period)); bool DoTrade = true; int MMIPeriod = optimize(0,200,500,100); if(MMIPeriod) { MMIPeriod = checkLookBack(MMIPeriod); var *MMI_Raw = series(MMI(Price,MMIPeriod)); var *MMI_Smooth = series(LowPass(MMI_Raw,MMIPeriod)); DoTrade = falling(MMI_Smooth); } if(DoTrade) { if(valley(Smoothed)) enterLong(); else if(peak(Smoothed)) enterShort(); } } }

The 10 trend trading scripts with the 10 different indicators remain unchanged, aside from now including **TrendMMI.c** instead of **Trend.c**. Trading is now dependent on a boolean variable **DoTrade**. The length of the MMI range is varied between 200, 300, 400, and 500 bars. As most parameters in a strategy, the MMI range is a compromise: It should be no less than 200 bars for getting any accuracy, but it should not be too long for preventing that different market regimes fall in the same MMI range. At the default range of 0, no MMI is applied and trading is not filtered. This way we’re including all the previous systems in the test. This is required for properly detecting Data Mining Bias, which must consider all systems that were discarded based on their result.

We’re running the MMI return value through a lowpass filter that uses the same period as the MMI range. This gives us a smooth MMI value that does not jump around. This value is now used for trade filtering: trades are opened and closed only when the smoothed MMI is falling, meaning that the market has entered trending mode within the last 200 to 500 bars. The MMI is only applied to one of the systems resulting from the prior period variation (the **optimize** function automatically selects the parameter of the “most robust” system before optimizing the next parameter). So now we’re testing in fact not 900, but 1260 systems: 900 without MMI and each 90 with MMI ranges of 200, 300, 400, and 500 bars. The systems with not enough trades or a too-long lookback period are again removed from the pool, so the real number of tested systems is about 1100.

Depending on the speed of your PC, Zorro will need about 1 hour to test all systems. At the end of every system test, Zorro produces the parameter histograms. We have now two parameters. The histogram of the first one, the price smoothing filter period, looks as before because MMI was switched off during optimization. The second histogram displays the MMI range in combination with the best value from the first histogram. “Best” is here not the highest bar from the previous histogram, but the value that Zorro deems the most robust and least sensitive to market changes. A typical MMI histograms look like this:

The first bar, marked “100”, is the best system without MMI. We can see that it is unprofitable: The profit factor (left scale) is only about 0.8. Using the MMI with a range of 200 and 300 makes the system in fact worse, and reduces the profit factor t0 0.7. However the last two MMI ranges, 400 and 500, shift the system into the profit zone. This was just a random example, but how does the MMI affect all the other systems? Here are the statistics from the MMI experiment:

Asset, Period, Indicator | Success Rate | Winning | Losing |

EUR/USD |
46% (+8%) |
154 |
185 |

S&P 500 |
4% (+3%) |
15 |
318 |

Silver |
27% (+7%) |
87 |
240 |

15 Minutes |
18% (+7%) |
71 |
322 |

1 Hour |
27% (+9%) |
92 |
251 |

4 Hours |
35% (+2%) |
93 |
170 |

ALMA |
22% (+6)% |
22 |
79 |

Decycle |
21% (+8%) |
23 |
89 |

EMA |
23% (+5%) |
24 |
79 |

HMA |
34% (+9%) |
33 |
66 |

Laguerre |
33% (+3%) |
20 |
38 |

LinearReg |
29% (+6%) |
31 |
77 |

Lowpass |
24% (+5%) |
26 |
82 |

SMA |
26% (+5%) |
27 |
76 |

Smooth |
26% (+7%) |
23 |
67 |

ZMA |
22% (+8)% |
27 |
90 |

The **Rate** column shows the percentage of successful systems, and in parentheses the difference to the percentage without MMI. We can see that the MMI increased the number of successful systems in all markets, time frames, and indicators. However the numbers are not really representative: the MMI only affected a quarter of the tested systems, but the upper quarter, so some increase in the number of profitable systems was to be expected anyway. A more meaningful measure is the WRC. We’re using the same **Bootstrap.c** script as in the previous experiment, we only need to increase the CURVES number to 1260. This is the WRC histogram of systems with MMI (again, best system vs. bootstrapped returns of all systems):

The MMI filter now shifted the best system (black) far to the right side of the histogram. It got a **p-value of 0.02**, meaning that it is better than 98% of the best randomized systems, and thus well above the 95% significance level. Using the MMI for filtering trades, the method of trading on curve peaks and valleys passed White’s Reality Check. In fact two of the 1260 systems got p-values above the significance level.

The best systems of the experiment had some things in common: They traded with silver and used either the ALMA or the lowpass filter. This is a surprising result, because neither silver nor ALMA and lowpass had the highest number of profitable systems. From the above table, one would assume that EUR/USD and the HMA or Laguerre filter are the most promising. They indeed produced many apparently good systems with profit factors above 2 (without trade costs), but none of them passed the WRC.

### Conclusion

- The MMI improves trend following systems with all tested markets, time frames, and indicators.
- Trend following systems using the MMI can pass White’s Reality Check.
- From the 10 tested smoothing indicators, ALMA produced the best results, although within a relatively small parameter range.
- To do: Test more trend filters, f.i. the Hurst Exponent or Ehlers’ Trend/ Cycle decomposition.
- To do: Create a real trading system by combining the best trend systems and adding the usual system components such as stop loss, trailing algorithm, profit lock, money management, and so on.

I’ve added the scripts to the 2015 scripts collection. Please note that you’ll again need Zorro 1.36 or above for reproducing the experiment.

Nice work jcl. Ciao