This indicator can improve – sometimes even double – the profit expectancy of trend following systems. The **Market Meanness Index** tells whether the market is currently moving in or out of a “trending” regime. It can this way prevent losses by **false signals** of trend indicators. It is a purely statistical algorithm and not based on volatility, trends, or cycles of the price curve.

There are already several methods for differentiating trending and nontrending market regimes. Some of them are rumored to really work, at least occasionally. John Ehlers proposed the **Hilbert Transform** or a **Cycle / Trend decomposition**, Benoit Mandelbrot the **Hurst Exponent**. In comparison, the source code of the Market Meanness Index is relatively simple:

// Market Meanness Index double MMI(double *Data,int Length) { double m = Median(Data,Length); int i, nh=0, nl=0; for(i=1; i<Length; i++) { if(Data[i] > m && Data[i] > Data[i-1]) // mind Data order: Data[0] is newest! nl++; else if(Data[i] < m && Data[i] < Data[i-1]) nh++; } return 100.*(nl+nh)/(Length-1); }

This code is in C for Zorro, but there’s also a MQL4 MMI version that someone on Steve Hopwood’s forum has programmed (see blogroll). As the name suggests, the indicator measures the meanness of the market – its tendency to revert to the mean after pretending to start a trend. If that happens too often, all trend following systems will bite the dust.

### The Three-Quarter Rule

A series of random numbers reverts to the mean – or more precisely, to the median – with a probability of 75%. So when you look at a random price sequence, if yesterday’s price was above the median, in 75% of all cases today’s price is lower than yesterday’s. And if the yesterday’s price was below the median, 75% chance is that today’s price is higher. The proof of the 75% is relatively simple and won’t require integral calculus. Consider a price curve with median **M**. By definition, half the prices are less than **M** and half are greater (for simplicity’s sake we’re ignoring the case when a price is exactly **M**). Now combine the prices of the curve to pairs each consisting of a price **Py** and the following price **Pt**. Thus each pair represents a price change from **Py** to **Pt**. We now got a lot of price changes that we divide into four sets:

**(Pt < M, Py < M)****(Pt < M, Py > M)****(Pt > M, Py < M)****(Pt > M, Py > M)**

These four sets have obviously the same number of elements – that is, 1/4 of all **Py->Pt** price changes – when **Pt** and **Py** are not correlated, i.e. completely independent of one another. The value of **M** and the shape of the price curve won’t matter for this. Now how many price pairs revert to the median? All pairs that fulfill this condition: **(Py < M and Pt > Py) or (Py > M and Pt < Py)** The condition in the first bracket is fulfilled for half the prices in set 1 (in the other half is **Pt** less than** Py**) and in the whole set 3 (because **Pt** is always higher than **Py** in set 3). So the first bracket is true for 1/2 * 1/4 + 1/4 = 3/8 of all price changes. Likewise, the second bracket is true in half the set 4 and in the whole set 2, thus also for 3/8 of all price changes. 3/8 + 3/8 yields 6/8, i.e. **75%**. This is the three-quarter rule for the differences of random numbers.

The **MMI** function just counts the number of data differences for which the conditition is met, and returns their percentage. The **Data** series may contain prices or price changes. Prices have always some serial correlation: If EUR / USD today is at 1.20, it will also be tomorrow around 1.20. That it will end up tomorrow at 70 cents or 2 dollars per EUR is rather unlikely. This serial correlation is also true for a price series calculated from random numbers, as not the prices themselves are random, but their changes. Thus, the MMI function should return a smaller percentage, such as 55%, when fed with prices.

Unlike prices, price changes have not necessarily serial correlation. A one hundred percent efficient market has no correlation between the price change from yesterday to today and the price change from today to tomorrow. If the MMI function is fed with perfectly random price changes from a perfectly efficient market, it will return a value of about 75%. The less efficient and the more trending the market becomes, the more the MMI decreases. Thus a falling MMI is a indicator of an upcoming trend. A rising MMI hints that the market will get nastier, at least for trend trading systems.

### Using the MMI in a trend strategy

One could assume that MMI predicts the price direction. A high MMI value indicates a high chance of mean reversion, so when prices were moving up in the last time and MMI is high, can we expect a soon price drop? Unfortunately it doesn’t work this way. The probability of mean reversion is not evenly distributed over the **Length** of the **Data** interval. For the early prices it is high (since the median is computed from future prices), but for the late prices, at the very time when MMI is calculated, it is down to just 50%. Predicting the next price with the MMI would work as well as flipping a coin.

Another mistake would be using the MMI for detecting a cyclic or mean-reverting market regime. True, the MMI will rise in such a situation, but it will also rise when the market becomes more random and more effective. A rising MMI alone is no promise of profit by cycle trading systems.

So the MMI won’t tell us the next price, and it won’t tell us if the market is mean reverting or just plain mean, but it can reveal information about the success chance of trend following. For this we’re making an assumption: **Trend itself is trending**. The market does not jump in and out of trend mode suddenly, but with some inertia. Thus, when we know that MMI is rising, we assume that the market is becoming more efficient, more random, more cyclic, more reversing or whatever, but in any case bad for trend trading. However when MMI is falling, chances are good that the next beginning trend will last longer than normal.

This way the MMI can be an excellent trend filter – in theory. But we all know that there’s often a large gap between theory and practice, especially in algorithmic trading. So I’m now going to test what the Market Meanness Index does to the collection of the 900 trend following systems that I’ve accumulated. For a first quick test, this was the equity curve of one of the systems, **TrendEMA**, without MMI (44% average annual return):

This is the same system with MMI (55% average annual return):

We can see that the profit has doubled, from $250 to $500. The profit factor climbed from 1.2 to 1.8, and the number of trades (green and red lines) is noticeable reduced. On the other hand, the equity curve started with a drawdown that wasn’t there with the original system. So MMI obviously does not improve all trades. And this was just a randomly selected system. If our assumption about trend trendiness is true, the indicator should have a significant effect also on the other 899 systems.

This experiment will be the topic of the next article, in about a week. As usually I’ll include all the source code for anyone to reproduce it. Will the MMI miserably fail? Or improve only a few systems, but worsen others? Or will it light up the way to the Holy Grail of trend strategies? Let the market be the judge.

There are some links in the page which does not work. As “next page” in some cases.

There are some indicators which try to find trend more. Did you consider to do an analysis of all of them as you did with the trend indicators. I mean instead of just using MMI to the 900 systems, you could use another ones by using the same approach.

I can certainly test other trend indicators, as this requires only a small change of the script. But I don’t know many that promise to really work. Which indicator do you have in mind?

I dont have any really in mind. You mention above Hurst and Hilbert Transform and so on. I like the methodology you use to find out which trend indicator is actually better. I was wondering what would be the result if you use the same methodology to find out if the market is in trend mode by using different indicators and not just MMI.

Yes, I’ll test more filters and also the more conventional crossover rather than peaks and valleys for trend systems. The MMI already produces pretty good results, but it can not harm to test as many algorithms as possible. However the next planned experiment is with a “deep learning” network.

Well congratulations again for this amazing blog. Really inspiring as well.

I wonder if you can keep aplying this kind of approach in the next steps of strategy development like one can lock the best filter found so far and use instead as a variable input different kind of money management functions for example.

Nice article; I’m looking forward to trying this out sometime. Also, I love the name of your blog.

Um, isnt there lookahead bias here? the median is computed over the entire data set. Not sure it can be used at any point in the data set to make a decision, as is.

Ah, you really looked in the code! If the median was computed over the entire data set, it would indeed be lookahead bias. But ‘Length’ in the code is only the length of the MMI period. You can not look ahead with Zorro, except when you set a certain flag, otherwise any access of future data produces a warning message.

Hey! I’m enjoying the blog

There seems to be an unnecessary restriction on your test for mean reversion vs trend. Why must the price be beyond the median before measuring which direction it goes? “Mean reversion” usually is referring to a more general ‘where prices really ought to be’ rather than the mathematical mean (or in your case, median). Wouldn’t a more appropriate trend-vs-mean-reversion system simply look at auto-correlation? If a price moves one direction and then back the other way, regardless of its relative position to the median, then it would be considered “reverting”. This doesn’t invalidate your approach, but insisting that noise is mean-reverting 75% of the time isn’t describing the reality accurately. Really you’re saying “noise moving away from the median tends to move back toward the median 75% of the time.” Or did I misunderstand?

Thanks!

I think you understood it correctly – your formulation is indeed more accurate. The MMI is not really intended for distinguishing between mean reversion and trend. For this f.i. the Hurst Exponent produces better results, at least according to my experiences. The MMI works however well for determining just the presence or absence of trend. As soon as it begins to return high values in the 75% area, it can be the begin of mean reversion or it can be just randomness – the MMI can not distinguish that. Both situations are likewise bad for trend following.

Based on your post I have programmed and tested MMI on Quantopian. I choose different stocks (SPY, BAC, AAPL, NFLX, etc) daily close price changes and different MMI lengths (100,200,300,500) . The testing date range was from 01/01/2012 to 01/14/2016.

The result was always around 75% There was no value under 70% and above 85% in any case.

For example:

SPY

from 01/01/2012 to 01/14/2016

MMI length 300

Average: 76.092%

Standard deviation: 1.716

Max:80.27%

Min:71.91%

Is this a good result?

It seems to me that MMI is not usable or just this small changes under 75% what I should follow?

If I used prices instead of price changes, the result was always around 50%, the standard deviation was higher, but not so convincing.

Yes, this is a good result when you tested daily returns, which are normally not significantly trending. Even a bit mean reverting before mid-2014. You should get such a curve:

But as soon as you look into hour returns, you’ll see more trendiness:

The script:

Thank you for your quick answer.

So all trend following systems based on daily close prices are doomed.

In case of 1H or forex my hypothesis is that these continuous, connected data in time and so more trendiness. Between daily close prices there are a big gap in time after the market closes and next day opens.

So maybe I should trade on shorter time scale or on forex.

“So when you look at a random price sequence, if yesterday’s price was above the median, in 75% of all cases today’s price is lower than yesterday”

In a random price sequence, the chance of going higher/lower at each point is *always* 50% – irrespective of where the mean lies or what happened yesterday.

This is indeed a frequent misconception. The chance of going higher/lower at a certain point is very different to the chance of a price pair in a random sequence to revert to the median. If you do not believe the proof above, just run the script and look at the results. And if you do not believe the script either, roll the dice a hundred times, write down all the numbers, and count how often subsequent numbers revert to the median. 🙂

“If you do not believe the proof above, just run the script and look at the results”

So the snippet below is a Python transcription of the above run over a random walk sequence.

The MMI is always 50% Why the difference?

import random

import numpy

def random_walk(n):

seq = [0]

for i in range(n):

if random.random() > 0.5:

seq.append( seq[-1] + 1 )

else:

seq.append( seq[-1] – 1 )

return seq

def market_meanness_index(data):

m = numpy.median(data)

nh = nl = 0

for i in range(1, len(data)):

if data[i] > m and data[i] > data[i-1]:

nl += 1

elif data[i] < m and data[i] < data[i-1]:

nh += 1

return 100.0 * (nl+nh) / (len(data)-1)

mmi = market_meanness_index(random_walk(10**6))

print "{:.1f}%".format(mmi)

Because your random walk is not a random sequence. A random walk has strong serial correlation. A correct random sequence would be:

def random_walk(n):

seq = [0]

for i in range(n):

seq.append(random.random())

return seq

The same goes for price data: prices are not random. They have serial correlation. But price changes are random, at least in a perfect efficient market.

That makes sense! Thanks for the clarification and the prompt responses.

Hi, can you please explain the detail algorithm that how MMI helps in trend detection? Or some pseudo code. Though I found you check “falling” in the other blog, I don’t know how “falling” works.

Thanks

I think the algorithm is explained above under “The three-quarter rule”. If something is unclear, just ask. The MMI detects trend indirectly, by absence of mean reversion. “Falling” is a binary function that just determines if a data series is rising or falling.

@Mathafarn and anyone who wants to have a look at MMI for Python, the python version shown is not the same as Zorros version.

First of all np.median() isn’t the same as the median Zorro uses, second the for-loop is going the wrong way leading to different results.

Here’s a fixed version:

def market_meanness_index(data):

data_sorted = sorted(data)

if len(data) % 2:

m = data_sorted[int(len(data) / 2)]

else:

m = (data_sorted[int(len(data_sorted) / 2)+1] + data_sorted[int(len(data_sorted) / 2)]) / 2.0

nh = nl = 0

for i in range(0, len(data)-1):

if data[i] > m and data[i] > data[i+1]:

nl += 1

elif data[i] < m and data[i] < data[i+1]:

nh += 1

return 100.0 * (nl+nh) / (len(data)-1)

Hi jcl,

Great job!

Several quick questions

1. About the application of MMI, can we use percentile(MMI,25), if today’s MMI is lower than 25%, we say market is in trend, otherwise, not.

2. Which ones are sound indicators for determining market regime?

3. Some markets are easier to generate profit from than others, I call them more “tradable”, just like what you found in this experiment that SPY is the most noisy market while commodities are more trendy, any indicators on this?

Thank you

Jeff

1. Yes, a percentile threshold would certainly make sense.

2. For instance the Hurst Exponent – a fellow blogger, Robot Wealth, has recently written an article about it.

3. Theoretically, an indicator like Shannon Entropy could be used for determining more randomness or more tradeability in a market. This could be an interesting topic for research.

The python example below tests a perfect trend and a perfect mean reversion data sequence. The MMI for both is 50%. Why is this? Thanks….

import math

import numpy

def market_meanness_index(data):

m = numpy.median(data)

nh = nl = 0

for i in range(1, len(data)):

if data[i] > m and data[i] > data[i-1]:

nl += 1

elif data[i] < m and data[i] < data[i-1]:

nh += 1

return 100.0*(nl+nh)/(len(data)-1)

# perfect trend: MMI 50%

trend_data = numpy.arange(0.0, math.pi, 0.01)

print market_meanness_index(trend_data)

# perfect mean reversion: MMI 50%

sin_data = map(lambda x: math.sin(x), trend_data)

print market_meanness_index(sin_data)

A flat trend will indeed produce about 50%, but a sine curve should give you a heavily fluctuating MMI in the 60% or 70% area, dependent on the period of the sine curve in relation to the MMI period.

`function run()`

{

MaxBars = 1500;

LookBack = 100;

asset(""); // dummy asset

vars Data1 = series(genSine(5));

plot("Sine",Data1,NEW,BLACK);

plot("MMI_Sine",MMI(Data1,100),NEW,BLACK);

`vars Data2 = series(Bar*0.01);`

plot("Trend",Data2,NEW,BLACK);

plot("MMI_Trend",MMI(Data2,100),NEW,BLACK);

}

Thanks for the excellent article. Would MMI be useful to prevent losses in am mean reverting strategy or which other indocator would you recommend? Thank you.

No, the MMI will probably not work well for filtering mean reverting trades, since it makes no difference between “less trendy” and “more random”. For filtering mean reversion I would try to detect the dominant cycle – mean reversion is normally related to some short-term cyclic behavior – and check the amplitude of the dominant frequency component.

Thanks, will look into that.

Hi jcl! Do you happen to have the MetaTrader version of this? Every time I try to access the link you posted I get an error message.

Also do you happen to know if anyone has coded this for Tradestation?

It’s some time ago, but if I remember right someone posted a MT4 version on Steve Hopwood’s forum. There was also a Tradestation version, but I cannot remember the link anymore. It should be anyway relatively easy to port the code to Tradestation.

I tried to translate the MMI into Easylanguage. But I always just get a steadily increasing line.

nl and nh are always increasing values?

I’m no Easylanguage expert, but AFAIK it has no local variables, anything is global. If so, then make sure that nl and nh are reset to zero at the begin of the function.

If I reset nl and nh at each bar, I get a binary wave between 0 and 0.33.

Do I have to loop the calculation? For how long?

You should get about 0.75 when you test with random numbers. With 0.33, something is wrong. The indicator is not cumulative, so you need not loop it.

if nl and nh get reset every bar and the indicator is not cumulative, the formula would always be

100*1/299

as nl or nh are always 1

Could you maybe paraphrase the formula in pseudo-code, in case I misread the C-code?

I don’t know if it helps, but anyway:

count all price decreases above the median

add all price increases below the median

divide by the number of prices and return the percentage.

Hacker’s first rule: Code only what you understand.

>>jcl says:

>>July 18, 2017 at 10:11

>>It’s some time ago, but if I remember right someone posted a MT4 version on Steve Hopwood’s forum.

>>There was also a Tradestation version, but I cannot remember the link anymore. It should be

>>anyway relatively easy to port the code to Tradestation.

The MT4 version was on this link, but it seems to be deleted:

http://www.stevehopwoodforex.com/phpBB3/viewtopic.php?uid=17&f=27&t=3714&start=0

Maybe someone still has the indicator for MT4 or Tradestation

I have translated the indicator, but it looks quite different from your example

http://imgur.com/a/WWD0O

calculated on close of daily bars, length 300

Seems like it’s high, when trending and low when mean reverting. Also highest value is 60%

Then apparently the translation was still not a real success. You must get the same curve, and trending produces lower, not higher values.

but the calculation base is correct? length 300 on daily close, compared to median?

Yes. But make sure that you get the data order right. It’s a financial series, with the newest data at the begin. If the order is wrong, rising and falling is swapped.

you mean it would be different if the counter of true conditions is increasing (from old to new data) from looking back from the current bar to the last 300 bars?

Look at the code. Changing the data order is equivalent to swapping two of the four ‘>’ and ‘<' operators.

Hi, I have a question about price changes being used in this indicator. I have found a version that uses open[0] – close[0] to represent the price change.

Is that accurate or should we use close[0] – close[1] as the price change?

Does it mater or could I use any highl0] – low[0].

Also I have seen this indicator coded incorrectly rather than using the median which is the middle of a price series some versions are using the average ( arithmetic mean). F

Will that use adversely effect the calculation?

close[0]-close[1] is the correct price change. It won’t make a difference for 24h traded assets, but can produce a different result for stocks. And the average is wrong. It must be really the median.

I’m backtesting MMI in a set of prices where I clearly see a downtrend. I’m getting an MMI of 40 if I test it with prices, and an MMI of 48 if I test it with price change. I guess that it means that the market is in a strong tendency. Am I right?

So, could we say that the lower the MMI, the better for trend trading systems? So if we find three markets with values 30, 20, and 10 , would be the last one the better for a trend trading strategy?

Well, I guess I talked too soon, sorry for that. Now, I’m trying to analyze MMI within a 30m strategy ; with a list of 404 price changes, completely sideways (https://imgur.com/a/ggwJ3), I’m getting a MMI of 50.49; If instead I use the close prices, I’m getting a MMI of 48.76. In this case obviously the price is not in a trend because it goes up & down.

An MMI of 48 is unusual, and means either a very strong trend in the tested period, or a wrong MMI implementation. On normal trending price curves the MMI is in the area of 70.

I’m going to paste the MMI code (python + pandas) also for the records, it may be useful for someone else (I think the code is ok but I may be wrong!). What I don’t get is why I’m getting a similar result (around 45-50) if the curve price is a “mountain” (trend up and then down) or if it is just an uptrend

def MMI(df):

m = df.Close.median()

length = len(df.index)

nh=0

nl=1

df.loc[0,”nl”] = 0

df.loc[0,”nh”] = 0

df[“nl”] = df[“nl”].fillna(0)

df[“nh”] = df[“nh”].fillna(0)

df[“nl”] = np.where((df.Close > m) & (df.Close > df.Close.shift(1)), 1, 0)

df[“nh”] = np.where((df.Close df.Close.shift(1)), 1, 0)

nl=df[“nl”].sum()

nh=df[“nh”].sum()

ret= 100.*(nl+nh)/(length-1);

return ret

nh and nl initialization in my code is useless, by the way (it was due to an old code). It is overwritten anyway.

You can easily test if your implementation works: apply it to random data. The result must then be 75. Look in the earlier comments, I remember that another user had a similar problem, also with a Python implementation.

It seems that my implementation is broken, my MMI generates 48 with random data…

Anyway, I’m testing the TrendMMI script in Zorro latest version, but when I train/execute it, it says “Undefined function called!” and it doesn’t generates any value while training, and when I click on Test, it only generates the price curve but nothing else. So I guess something has changed since 2015 and now in Zorro that makes this fail. Can this be fixed somehow?

There are many scripts in the 2015 archive that you can train or execute, but TrendMMI does not belong to them. It is not a strategy, but a function library for including.

Shame on me, this was a n00b problem! Ok, fixed that, thanks.

Another error shows when Training some of the scripts; TrendDecycle or TrendEMA for instance: “Error 046, H4 LookBack exceeded by 16 bars (7696)”. I think I “fixed” it changing this line in TrendMMI.c:

int MMIPeriod = optimize (0,200,500,100)

with:

int MMIPeriod = optimize (0,200,450,100)

You’re right, the MMI scripts were outdated. I had modified them in 2016, but for some reason the modification did not make it to my server. I’ve now uploaded the right scripts.

Could you please post a screenshot of the MMI for daily bars for a usual commodity (like Gold or Crude Oil…), so we can compare, if other versions of it are build correctly?

Sure:

`void run()`

{

vars Returns = series(priceClose(0)-priceClose(1));

plot("MMI",MMI(Returns,300),NEW,RED);

}