This indicator can improve – sometimes even double – the profit expectancy of trend following systems. The **Market Meanness Index** tells whether the market is currently moving in or out of a “trending” regime. It can this way prevent losses by **false signals** of trend indicators. It is a purely statistical algorithm and not based on volatility, trends, or cycles of the price curve.

There are already several methods for differentiating trending and nontrending market regimes. Some of them are rumored to really work, at least occasionally. John Ehlers proposed the **Hilbert Transform** or a **Cycle / Trend decomposition**, Benoit Mandelbrot the **Hurst Exponent**. In comparison, the source code of the Market Meanness Index is relatively simple:

var MMI(var *Data,int Length) { var m = Median(Data,Length); int i, nh=0, nl=0; for(i=1; i<Length; i++) { if(Data[i] > m && Data[i] > Data[i-1]) nl++; else if(Data[i] < m && Data[i] < Data[i-1]) nh++; } return 100.*(nl+nh)/(Length-1); }

This code is for Zorro, but there’s also a MMI version for MetaTrader4® that someone on Steve Hopwood’s forum has programmed – the source code can be downloaded there. As the name suggests, the indicator measures the meanness of the market – its tendency to revert to the mean after pretending to start a trend. If that happens too often, all trend following systems will bite the dust.

### The Three-Quarter Rule

A series of random numbers reverts to the mean – or more precisely, to the median – with a probability of 75%. So when you look at a random price sequence, if yesterday’s price was above the median, in 75% of all cases today’s price is lower than yesterday’s. And if the yesterday’s price was below the median, 75% chance is that today’s price is higher. The proof of the 75% is relatively simple and won’t require integral calculus. Consider a price curve with median **M**. By definition, half the prices are less than **M** and half are greater (for simplicity’s sake we’re ignoring the case when a price is exactly **M**). Now combine the prices of the curve to pairs each consisting of a price **Py** and the following price **Pt**. Thus each pair represents a price change from **Py** to **Pt**. We now got a lot of price changes that we divide into four sets:

**(Pt < M, Py < M)****(Pt < M, Py > M)****(Pt > M, Py < M)****(Pt > M, Py > M)**

These four sets have obviously the same number of elements – that is, 1/4 of all **Py->Pt** price changes – when **Pt** and **Py** are not correlated, i.e. completely independent of one another. The value of **M** and the shape of the price curve won’t matter for this. Now how many price pairs revert to the median? All pairs that fulfill this condition: **(Py < M and Pt > Py) or (Py > M and Pt < Py)** The condition in the first bracket is fulfilled for half the prices in set 1 (in the other half is **Pt** less than** Py**) and in the whole set 3 (because **Pt** is always higher than **Py** in set 3). So the first bracket is true for 1/2 * 1/4 + 1/4 = 3/8 of all price changes. Likewise, the second bracket is true in half the set 4 and in the whole set 2, thus also for 3/8 of all price changes. 3/8 + 3/8 yields 6/8, i.e. **75%**. This is the three-quarter rule for the differences of random numbers.

The **MMI** function just counts the number of data differences for which the conditition is met, and returns their percentage. The **Data** series may contain prices or price changes. Prices have always some serial correlation: If EUR / USD today is at 1.20, it will also be tomorrow around 1.20. That it will end up tomorrow at 70 cents or 2 dollars per EUR is rather unlikely. This serial correlation is also true for a price series calculated from random numbers, as not the prices themselves are random, but their changes. Thus, the MMI function should return a smaller percentage, such as 55%, when fed with prices.

Unlike prices, price changes have not necessarily serial correlation. A one hundred percent efficient market has no correlation between the price change from yesterday to today and the price change from today to tomorrow. If the MMI function is fed with perfectly random price changes from a perfectly efficient market, it will return a value of about 75%. The less efficient and the more trending the market becomes, the more the MMI decreases. Thus a falling MMI is a indicator of an upcoming trend. A rising MMI hints that the market will get nastier, at least for trend trading systems.

### Using the MMI in a trend strategy

One could assume that MMI predicts the price direction. A high MMI value indicates a high chance of mean reversion, so when prices were moving up in the last time and MMI is high, can we expect a soon price drop? Unfortunately it doesn’t work this way. The probability of mean reversion is not evenly distributed over the **Length** of the **Data** interval. For the early prices it is high (since the median is computed from future prices), but for the late prices, at the very time when MMI is calculated, it is down to just 50%. Predicting the next price with the MMI would work as well as flipping a coin.

Another mistake would be using the MMI for detecting a cyclic or mean-reverting market regime. True, the MMI will rise in such a situation, but it will also rise when the market becomes more random and more effective. A rising MMI alone is no promise of profit by cycle trading systems.

So the MMI won’t tell us the next price, and it won’t tell us if the market is mean reverting or just plain mean, but it can reveal information about the success chance of trend following. For this we’re making an assumption: **Trend itself is trending**. The market does not jump in and out of trend mode suddenly, but with some inertia. Thus, when we know that MMI is rising, we assume that the market is becoming more efficient, more random, more cyclic, more reversing or whatever, but in any case bad for trend trading. However when MMI is falling, chances are good that the next beginning trend will last longer than normal.

This way the MMI can be an excellent trend filter – in theory. But we all know that there’s often a large gap between theory and practice, especially in algorithmic trading. So I’m now going to test what the Market Meanness Index does to the collection of the 900 trend following systems that I’ve accumulated. For a first quick test, this was the equity curve of one of the systems, **TrendEMA**, without MMI (44% average annual return):

This is the same system with MMI (55% average annual return):

We can see that the profit has doubled, from $250 to $500. The profit factor climbed from 1.2 to 1.8, and the number of trades (green and red lines) is noticeable reduced. On the other hand, the equity curve started with a drawdown that wasn’t there with the original system. So MMI obviously does not improve all trades. And this was just a randomly selected system. If our assumption about trend trendiness is true, the indicator should have a significant effect also on the other 899 systems.

This experiment will be the topic of the next article, in about a week. As usually I’ll include all the source code for anyone to reproduce it. Will the MMI miserably fail? Or improve only a few systems, but worsen others? Or will it light up the way to the Holy Grail of trend strategies? Let the market be the judge.

There are some links in the page which does not work. As “next page” in some cases.

There are some indicators which try to find trend more. Did you consider to do an analysis of all of them as you did with the trend indicators. I mean instead of just using MMI to the 900 systems, you could use another ones by using the same approach.

I can certainly test other trend indicators, as this requires only a small change of the script. But I don’t know many that promise to really work. Which indicator do you have in mind?

I dont have any really in mind. You mention above Hurst and Hilbert Transform and so on. I like the methodology you use to find out which trend indicator is actually better. I was wondering what would be the result if you use the same methodology to find out if the market is in trend mode by using different indicators and not just MMI.

Yes, I’ll test more filters and also the more conventional crossover rather than peaks and valleys for trend systems. The MMI already produces pretty good results, but it can not harm to test as many algorithms as possible. However the next planned experiment is with a “deep learning” network.

Well congratulations again for this amazing blog. Really inspiring as well.

I wonder if you can keep aplying this kind of approach in the next steps of strategy development like one can lock the best filter found so far and use instead as a variable input different kind of money management functions for example.

Nice article; I’m looking forward to trying this out sometime. Also, I love the name of your blog.

Um, isnt there lookahead bias here? the median is computed over the entire data set. Not sure it can be used at any point in the data set to make a decision, as is.

Ah, you really looked in the code! If the median was computed over the entire data set, it would indeed be lookahead bias. But ‘Length’ in the code is only the length of the MMI period. You can not look ahead with Zorro, except when you set a certain flag, otherwise any access of future data produces a warning message.

Hey! I’m enjoying the blog

There seems to be an unnecessary restriction on your test for mean reversion vs trend. Why must the price be beyond the median before measuring which direction it goes? “Mean reversion” usually is referring to a more general ‘where prices really ought to be’ rather than the mathematical mean (or in your case, median). Wouldn’t a more appropriate trend-vs-mean-reversion system simply look at auto-correlation? If a price moves one direction and then back the other way, regardless of its relative position to the median, then it would be considered “reverting”. This doesn’t invalidate your approach, but insisting that noise is mean-reverting 75% of the time isn’t describing the reality accurately. Really you’re saying “noise moving away from the median tends to move back toward the median 75% of the time.” Or did I misunderstand?

Thanks!

I think you understood it correctly – your formulation is indeed more accurate. The MMI is not really intended for distinguishing between mean reversion and trend. For this f.i. the Hurst Exponent produces better results, at least according to my experiences. The MMI works however well for determining just the presence or absence of trend. As soon as it begins to return high values in the 75% area, it can be the begin of mean reversion or it can be just randomness – the MMI can not distinguish that. Both situations are likewise bad for trend following.

Based on your post I have programmed and tested MMI on Quantopian. I choose different stocks (SPY, BAC, AAPL, NFLX, etc) daily close price changes and different MMI lengths (100,200,300,500) . The testing date range was from 01/01/2012 to 01/14/2016.

The result was always around 75% There was no value under 70% and above 85% in any case.

For example:

SPY

from 01/01/2012 to 01/14/2016

MMI length 300

Average: 76.092%

Standard deviation: 1.716

Max:80.27%

Min:71.91%

Is this a good result?

It seems to me that MMI is not usable or just this small changes under 75% what I should follow?

If I used prices instead of price changes, the result was always around 50%, the standard deviation was higher, but not so convincing.

Yes, this is a good result when you tested daily returns, which are normally not significantly trending. Even a bit mean reverting before mid-2014. You should get such a curve:

But as soon as you look into hour returns, you’ll see more trendiness:

The script:

Thank you for your quick answer.

So all trend following systems based on daily close prices are doomed.

In case of 1H or forex my hypothesis is that these continuous, connected data in time and so more trendiness. Between daily close prices there are a big gap in time after the market closes and next day opens.

So maybe I should trade on shorter time scale or on forex.

“So when you look at a random price sequence, if yesterday’s price was above the median, in 75% of all cases today’s price is lower than yesterday”

In a random price sequence, the chance of going higher/lower at each point is *always* 50% – irrespective of where the mean lies or what happened yesterday.

This is indeed a frequent misconception. The chance of going higher/lower at a certain point is very different to the chance of a price pair in a random sequence to revert to the median. If you do not believe the proof above, just run the script and look at the results. And if you do not believe the script either, roll the dice a hundred times, write down all the numbers, and count how often subsequent numbers revert to the median. 🙂

“If you do not believe the proof above, just run the script and look at the results”

So the snippet below is a Python transcription of the above run over a random walk sequence.

The MMI is always 50% Why the difference?

import random

import numpy

def random_walk(n):

seq = [0]

for i in range(n):

if random.random() > 0.5:

seq.append( seq[-1] + 1 )

else:

seq.append( seq[-1] – 1 )

return seq

def market_meanness_index(data):

m = numpy.median(data)

nh = nl = 0

for i in range(1, len(data)):

if data[i] > m and data[i] > data[i-1]:

nl += 1

elif data[i] < m and data[i] < data[i-1]:

nh += 1

return 100.0 * (nl+nh) / (len(data)-1)

mmi = market_meanness_index(random_walk(10**6))

print "{:.1f}%".format(mmi)

Because your random walk is not a random sequence. A random walk has strong serial correlation. A correct random sequence would be:

def random_walk(n):

seq = [0]

for i in range(n):

seq.append(random.random())

return seq

The same goes for price data: prices are not random. They have serial correlation. But price changes are random, at least in a perfect efficient market.

That makes sense! Thanks for the clarification and the prompt responses.

Hi, can you please explain the detail algorithm that how MMI helps in trend detection? Or some pseudo code. Though I found you check “falling” in the other blog, I don’t know how “falling” works.

Thanks

I think the algorithm is explained above under “The three-quarter rule”. If something is unclear, just ask. The MMI detects trend indirectly, by absence of mean reversion. “Falling” is a binary function that just determines if a data series is rising or falling.

@Mathafarn and anyone who wants to have a look at MMI for Python, the python version shown is not the same as Zorros version.

First of all np.median() isn’t the same as the median Zorro uses, second the for-loop is going the wrong way leading to different results.

Here’s a fixed version:

def market_meanness_index(data):

data_sorted = sorted(data)

if len(data) % 2:

m = data_sorted[int(len(data) / 2)]

else:

m = (data_sorted[int(len(data_sorted) / 2)+1] + data_sorted[int(len(data_sorted) / 2)]) / 2.0

nh = nl = 0

for i in range(0, len(data)-1):

if data[i] > m and data[i] > data[i+1]:

nl += 1

elif data[i] < m and data[i] < data[i+1]:

nh += 1

return 100.0 * (nl+nh) / (len(data)-1)

Hi jcl,

Great job!

Several quick questions

1. About the application of MMI, can we use percentile(MMI,25), if today’s MMI is lower than 25%, we say market is in trend, otherwise, not.

2. Which ones are sound indicators for determining market regime?

3. Some markets are easier to generate profit from than others, I call them more “tradable”, just like what you found in this experiment that SPY is the most noisy market while commodities are more trendy, any indicators on this?

Thank you

Jeff

1. Yes, a percentile threshold would certainly make sense.

2. For instance the Hurst Exponent – a fellow blogger, Robot Wealth, has recently written an article about it.

3. Theoretically, an indicator like Shannon Entropy could be used for determining more randomness or more tradeability in a market. This could be an interesting topic for research.