Most trading systems are of the get-rich-quick type. They exploit temporary market inefficiencies and aim for annual returns in the 100% area. They require regular supervision and adaption to market conditions, and still have a limited lifetime. Their expiration is often accompanied by large losses. But what if you’ve nevertheless collected some handsome gains, and now want to park them in a more safe haven? Put the money under the pillow? Take it into the bank? Give it to a hedge funds? Obviously, all that goes against an algo trader’s honor code. Here’s an alternative.

The old-fashioned investing method is buying some low-risk stocks and then waiting a long time. Any portfolio of stocks has a certain mean return and a certain fluctuation in value; you normally want to minimize the latter and maximize the former. The **optimal capital allocation** among the portfolio components produces either maximum mean return for a given allowed fluctuation, or minimum fluctuation – respectively, minimum variance – for a given mean return. This optimal allocation is often very different to investing the same amount in all N components of the portfolio. An easy way to solve this mean / variance optimization problem was published 60 years ago by **Harry Markowitz**. It won him later the Nobel prize.

### The unfashionable Markowitz

Unfortunately, Markowitz got largely out of fashion since then. The problem is the same as with all trading algorithms: You can only calculate the optimal capital allocation in hindsight. Optimized portfolios mysteriously failed in live trading. They were said to often return less than a simple 1/N capital distribution. But this was challenged recently in an interesting paper (1) by Keller, Butler, and Kipnis, of which I quote the first paragraph:

*Mean-Variance Optimization (MVO) as introduced by Markowitz (1952) is often presented as an elegant but impractical theory. MVO is an “unstable and error-maximizing” procedure (Michaud 1989), and is “nearly always beaten by simple 1/N portfolios” (DeMiguel, 2007). And to quote Ang (2014): “Mean-variance weights perform horribly… The optimal mean-variance portfolio is a complex function of estimated means, volatilities, and correlations of asset returns. There are many parameters to estimate. Optimized mean-variance portfolios can blow up when there are tiny errors in any of these inputs…”.*

MVO is sensitive to small changes in asset returns, but that does not yet mean that it won’t work. And whatever Ang (2014) meant with the “many parameters to estimate”, there is no such parameter in Markowitz’ algorithm. But the optimized portfolios by the quoted authors indeed blew up. The reason according to (1): The Markowitz vituperators did it all wrong. They used too long, mean-reverting time periods for sampling the returns and covariances, and they applied the MVO algorithm wrongly to mixed long/short portfolios. When correctly applied to a momentum-governed time period and long-only portfolios, MVO in fact produces out of sample results **far superior to 1/N**. This was proven by testing a number of example portfolios in (1) with a R MVO implementation by fellow blogger Ilya Kipnis.

However, a R implementation is not very practical for live trading. For this we have to implement MVO in a real trade platform. Then we can park our money in an optimized portfolio of stocks and ETFs, let the platform rebalance the capital allocation in regular intervals, lean back, wait, and get rich slowly.

### Implementing MVO

The Zorro implementation is based on Markowitz’ 1959 publication (2). In chapter 8, he described the MVO algorithm in a clear and easy to follow way. For simple minded programmers like me, he even included a brief introduction to linear algebra! I only modified his original algorithm by adding a weight constraint as suggested in (1). It turned out later that this constraint was a good idea – it stabilizes the algorithm and has a large positive effect on performance.

In wise anticipation of future computing machines, Markowitz also included an example portfolio for checking if you programmed his algorithm correctly. The proof:

function main() { var Means[3] = { .062,.146,.128 }; var Covariances[3][3] = { .0146,.0187,.0145,.0187,.0854,.0104,.0145,.0104,.0289 }; var Weights[3]; var BestVariance = markowitz(Covariances,Means,3); markowitzReturn(Weights,1); printf("\nMax: %.2f %.2f %.2f",Weights[0],Weights[1],Weights[2]); markowitzReturn(Weights,BestVariance); printf("\nBest: %.2f %.2f %.2f",Weights[0],Weights[1],Weights[2]); markowitzReturn(Weights,0); printf("\nMin: %.2f %.2f %.2f",Weights[0],Weights[1],Weights[2]); }

The means and covariances arrays in the script are from Markowitz’ example portfolio. The **markowitz** function runs the algorithm and returns the variance value associated with the best Sharpe ratio. The **markowitzReturn** function then calculates the capital allocation weights with the maximum mean return for a given variance. The weights for maximum, best, and minimum variance are printed. If I did it right, they should be exactly the same as in Markowitz’ publication:

Max: 0.00 1.00 0.00 Best: 0.00 0.22 0.78 Min: 0.99 0.00 0.01

### Selecting the assets

For long-term portfolios you can’t use the same high-leverage Forex or CFD instruments that you preferred for your short-term strategies. Instead you normally invest in stocks, ETFs, or similar instruments. They offer several advantages for algo trading:

**No zero-sum game.**In the long run, stocks and index ETFs have positive mean returns due to dividends and accumulated value, while Forex pairs and index CFDs have negative mean returns due to swap/rollover fees.

**Serious brokers.**Stock/ETF brokers are all regulated, what can not be said of all Forex/CFD brokers.

**More data**for your algorithms, such as volume and market depth information.

**Bigger choice of assets**from many different market sectors.

**More trading methods,**such as pairs trading (“stat arb”), trading risk-free assets such as T-bills, or trading volatility.

The obvious disadvantage is low leverage, like 1:4 compared with 1:100 or more for Forex instruments. Low leverage is ok for a long-term system, but not for getting rich quick. More restrictions apply to long-term portfolios. MVO obviously won’t work well with components that have no positive mean return. And it won’t work well either when the returns are strongly correlated. So when selecting assets for your long-term portfolio, you have to look not only for returns, but also for correlation. Here’s the main part of a Zorro script for that:

#define NN 30 // max number of assets function run() { BarPeriod = 1440; NumYears = 7; LookBack = 6*252; // 6 years string Names[NN]; vars Returns[NN]; var Correlations[NN][NN]; int N = 0; while(Names[N] = loop( "TLT","LQD","SPY","GLD","VGLT","AOK")) { if(is(INITRUN)) assetHistory(Names[N],FROM_YAHOO); asset(Names[N]); Returns[N] = series((priceClose(0)-priceClose(1))/priceClose(1)); if(N++ >= NN) break; } if(is(EXITRUN)) { int i,j; for(i=0; i<N; i++) for(j=0; j<N; j++) Correlations[N*i+j] = Correlation(Returns[i],Returns[j],LookBack); plotHeatmap("Correlation",Correlations,N,N); for(i=0; i<N; i++) printf("\n%i - %s: Mean %.2f%% Variance %.2f%%", i+1,Names[i], 100*annual(Moment(Returns[i],LookBack,1)), 252*100*Moment(Returns[i],LookBack,2)); } }

The script first sets up some parameters, then goes into a loop over **N** assets. Here I’ve just entered some popular ETFs; for replacing them, websites such as etfdb.com give an overview and help searching for the optimal ETF combination.

In the initial run, the asset prices are downloaded from Yahoo. They are corrected for splits and dividends. The **assetHistory** function stores them as historical price data files. Then the assets are selected and their returns are calculated and stored in the **Returns** data series. This is repeated with all 1-day bars of a 7 years test period (obviously the period depends on since when the selected ETFs are available). In the final run the script prints the annual mean returns and variances of all assets, which are the first and second **moments** of the return series. The **annual** function and the 252 multiplication factor convert daily values to annual values. The results for the selected ETFs:

1 - TLT: Mean 10.75% Variance 2.29% 2 - LQD: Mean 6.46% Variance 0.31% 3 - SPY: Mean 13.51% Variance 2.51% 4 - GLD: Mean 3.25% Variance 3.04% 5 - VGLT: Mean 9.83% Variance 1.65% 6 - AOK: Mean 4.70% Variance 0.23%

The ideal ETF has high mean return, low variance, and low correlation to all other assets of the portfolio. The correlation can be seen in the **correlation matrix** that is computed from all collected returns in the above code, then plotted in a **N*N** heatmap:

The correlation matrix contains the correlation coefficients of every asset with every other asset. The rows and columns of the heatmap are the 6 assets. The colors go from blue for low correlation between the row and column asset, to red for high correlation. Since any asset correlates perfectly with itself, we always have a red diagonal. But you can see from the other red squares that some of my 6 popular ETFs were no good choice. Finding the perfect ETF combination, with the heatmap as blue as possible, is left as an exercise to the reader.

### The efficient frontier

After selecting the assets for our portfolio, we now have to calculate the optimal capital allocation, using the MVO algorithm. However, “optimal” depends on the desired risk, i.e. volatility of the portfolio. For every risk value there’s a optimal allocation that generates the maximum return. So the optimal allocation is not a point, but a curve in the return / variance plane, named the **Efficient Frontier**. We can calculate and plot it with this script:

function run() { ... // similar to Heatmap script if(is(EXITRUN)) { int i,j; for(i=0; i<N; i++) { Means[i] = Moment(Returns[i],LookBack,1); for(j=0; j<N; j++) Covariances[N*i+j] = Covariance(Returns[i],Returns[j],LookBack); } var BestV = markowitz(Covariances,Means,N); var MinV = markowitzVariance(0,0); var MaxV = markowitzVariance(0,1); int Steps = 50; for(i=0; i<Steps; i++) { var V = MinV + i*(MaxV-MinV)/Steps; var R = markowitzReturn(0,V); plotBar("Frontier",i,V,100*R,LINE|LBL2,BLACK); } plotGraph("Max Sharpe",(BestV-MinV)*Steps/(MaxV-MinV), 100*markowitzReturn(0,BestV),SQUARE,GREEN); } }

I’ve omitted the first part since it’s identical to the heatmap script. Only the covariance matrix is now calculated instead of the correlation matrix. Covariances and mean returns are fed to the **markowitz** function that again returns the variance with the best Sharpe ratio. The subsequent calls to **markowitzVariance** also return the highest and the lowest variance of the efficient frontier and establish the borders of the plot. Finally the script plots 50 points of the annual mean return from the lowest to the highest variance:

At the right side we can see that the portfolio reaches a maximum annual return of about 12.9%, which is simply all capital allocated to SPY. On the left side we achieve only 5.4% return, but with less than a tenth of the daily variance. The green dot is the point on the frontier with the best Sharpe ratio (= return divided by square root of variance) at 10% annual return and 0.025 variance. This is the optimal portfolio – at least in hindsight.

### Experiments

How will a mean / variance optimized portfolio fare in an out of sample test, compared with with 1/N? Here’s a script for experiments with different portfolio compositions, lookback periods, weight constraints, and variances:

#define DAYS 252 // 1 year lookback period #define NN 30 // max number of assets function run() { ... // similar to Heatmap script int i,j; static var BestVariance = 0; if(tdm() == 1 && !is(LOOKBACK)) { for(i=0; i<N; i++) { Means[i] = Moment(Returns[i],LookBack,1); for(j=0; j<N; j++) Covariances[N*i+j] = Covariance(Returns[i],Returns[j],LookBack); } BestVariance = markowitz(Covariances,Means,N,0.5); } var Weights[NN]; static var Return, ReturnN, ReturnMax, ReturnBest, ReturnMin; if(is(LOOKBACK)) { Month = 0; ReturnN = ReturnMax = ReturnBest = ReturnMin = 0; } if(BestVariance > 0) { for(Return=0,i=0; i<N; i++) Return += (Returns[i])[0]/N; // 1/N ReturnN = (ReturnN+1)*(Return+1)-1; markowitzReturn(Weights,0); // min variance for(Return=0,i=0; i<N; i++) Return += Weights[i]*(Returns[i])[0]; ReturnMin = (ReturnMin+1)*(Return+1)-1; markowitzReturn(Weights,1); // max return for(Return=0,i=0; i<N; i++) Return += Weights[i]*(Returns[i])[0]; ReturnMax = (ReturnMax+1)*(Return+1)-1; markowitzReturn(Weights,BestVariance); // max Sharpe for(Return=0,i=0; i<N; i++) Return += Weights[i]*(Returns[i])[0]; ReturnBest = (ReturnBest+1)*(Return+1)-1; plot("1/N",100*ReturnN,AXIS2,BLACK); plot("Max Sharpe",100*ReturnBest,AXIS2,GREEN); plot("Max Return",100*ReturnMax,AXIS2,RED); plot("Min Variance",100*ReturnMin,AXIS2,BLUE); } }

The script goes through 7 years of historical data, and stores the daily returns in the **Returns** data series. At the first trading day of every month (**tdm() == 1**) it computes the means and the covariance matrix of the last 252 days, then calculates the efficient frontier. This time we also apply a 0.5 weight constraint to the minimum variance point. Based on this efficient frontier, we compute the daily total return with equal weights (**ReturnN**), best Sharpe ratio (**ReturnBest**), minimum variance (**ReturnMin**) and maximum Return (**ReturnMax**). The weights remain unchanged until the next rebalancing, this way establishing an out of sample test. The four daily returns are added up to 4 different equity curves :

We can see that MVO improves the portfolio in all three variants, in spite of its bad reputation. The black line is the 1/N portfolio with equal weights for all asset. The blue line is the minimum variance portfolio – we can see that it produces slightly better profits than 1/N, but with much lower volatility. The red line is the maximum return portfolio with the best profit, but high volatility and sharp drawdowns. The green line, the maximum Sharpe portfolio, is somewhere inbetween. Different portfolio compositions can produce a different order of lines, but the blue and green lines have almost always a much better Sharpe ratio than the black line. Since the minimum variance portfolio can be traded with higher leverage due to the smaller drawdowns, it often produces the highest profits.

For checking the monthly rebalancing of the capital allocation weights, we can display the weights in a heatmap:

The horizontal axis is the month of the simulation, the vertical axis the asset number. High weights are red and low weights are blue. The weight distribution above is for the maximum Sharpe portfolio of the 6 ETFs.

### The final money parking system

After all those experiments we can now code our long-term system. It shall work in the following way:

- The efficient frontier is calculated from daily returns of the last 252 trading days, i.e. one year. That’s a good time period for MVO according to (1), since most ETFs show 1-year momentum.

- The system rebalances the portfolio once per month. Shorter time periods, such as daily or weekly rebalancing, showed no advantage in my tests, but reduced the profit due to higher trading costs. Longer time periods, such as 3 months, let the system deteriorate.

- The point on the efficient frontier can be set up with a slider between minimum variance and maximum Sharpe. This way you can control the risk of the system.

- We use a 50% weight constraint at minimum variance. It’s then not anymore the optimal portfolio, but according to (1) – and my tests have confirmed this – it often improves the out of sample balance due to better diversification.

Here’s the script:

#define LEVERAGE 4 // 1:4 leverage #define DAYS 252 // 1 year #define NN 30 // max number of assets function run() { BarPeriod = 1440; LookBack = DAYS; string Names[NN]; vars Returns[NN]; var Means[NN]; var Covariances[NN][NN]; var Weights[NN]; var TotalCapital = slider(1,1000,0,10000,"Capital","Total capital to distribute"); var VFactor = slider(2,10,0,100,"Risk","Variance factor"); int N = 0; while(Names[N] = loop( "TLT","LQD","SPY","GLD","VGLT","AOK")) { if(is(INITRUN)) assetHistory(Names[N],FROM_YAHOO); asset(Names[N]); Returns[N] = series((priceClose(0)-priceClose(1))/priceClose(1)); if(N++ >= NN) break; } if(is(EXITRUN)) { int i,j; for(i=0; i<N; i++) { Means[i] = Moment(Returns[i],LookBack,1); for(j=0; j<N; j++) Covariances[N*i+j] = Covariance(Returns[i],Returns[j],LookBack); } var BestVariance = markowitz(Covariances,Means,N,0.5); var MinVariance = markowitzVariance(0,0); markowitzReturn(Weights,MinVariance+VFactor/100.*(BestVariance-MinVariance)); for(i=0; i<N; i++) { asset(Names[i]); MarginCost = priceClose()/LEVERAGE; int Position = TotalCapital*Weights[i]/MarginCost; printf("\n%s: %d Contracts at %.0f$",Names[i],Position,priceClose()); } } }

On Zorro’s panel you can set up the invested capital with a slider (**TotalCapital**) between 0 and 10,000$. A second slider (**VFactor**) is for setting up the desired risk from 0 to 100%: At 0 you’re trading with minimum variance, at 100 with maximum Sharpe ratio.

This script advises only, but does not trade: For automated trading it, you would need an API plugin to a ETF broker, such as IB. But the free Zorro version only has plugins for Forex/CFD brokers; the IB plugin is not free. However, since positions are only opened or closed once per month and price data is free from Yahoo, you do not really need an API connection for trading a MVO portfolio. Just fire up the above script once every month, and check what it prints out:

TLT: 0 Contracts at 129$ LQD: 0 Contracts at 120$ SPY: 3 Contracts at 206$ GLD: 16 Contracts at 124$ VGLT: 15 Contracts at 80$ AOK: 0 Contracts at 32$

Apparently, the optimal portfolio for this month consists of 3 contracts SPY, 16 contracts GLD, and 15 VGLT contracts. You can now manually open or close those positions in your broker’s trading platform until your portfolio matches the printed advice. Leverage is 4 by default, but you can change this to your broker’s leverage in the #define at the begin of the script. For a script that trades, simply replace the **printf** statement with a trade command that opens or closes the difference to the current position of the asset. This, too, is left as an exercise to the reader…

### MVO vs. OptimalF

It seems natural to use MVO not only for a portfolio of many assets, but also for a portfolio of many trading systems. I’ve tested this with the Z12 system that comes with Zorro and contains about 100 different system/asset combinations. It turned out that MVO did not produce better results than Ralph Vince’s **OptimalF** factors that are originally used by the system. OptimalF factors do not consider correlations between components, but they do consider the drawdown depths, while MVO is only based on means and covariances. The ultimate solution for such a portfolio of many trading systems might be a combination of MVO for the capital distribution and OptimalF for weight constraints. I have not tested this yet, but it’s on my to do list.

I’ve added all scripts to the 2016 script repository. You’ll need Zorro 1.44 or above for running them. When your fortune grows, reinvest only the **square root** of the growth (read here why). And after you made your first million with the MVO script, don’t forget to sponsor Zorro generously! 🙂

### Papers

- Momentum and Markowitz – A Golden Combination: Keller.Butler.Kipnis.2015

- Harry M. Markowitz, Portfolio Selection, Wiley 1959

There is no such metrics as to my knowledge, but you can easily invent some. For instance the sum of all elements of the correlation matrix divided by the number of elements.

Thank you for your reply.

If I find some new assets with higher mean/sqrt of the variance ratio than the existing one, should I just replace the old one with the new one right way? Or should we modify the assets list periodically? Like every year? Or we should not modify the asset list at all!

Have a good one!

By all means feel free to use other assets. From what I hear, most people use their own asset lists with the Z8 strategy, instead of the default assets.

Thank you for your reply but it’s not what I asked. I understand we can use other assets than AssetZ8, my question is: should we periodically, let’s say every year, update the assets list, using new assets with higher momentum, lower variance to replace the old ones which has been in the list. F.I., using TSLA, which is not in the list right now but with higher momentum and lower variance to replace GE, which has been in the list. Or we simple add the new ones to the list, then we have a longer list.

Hopefully I made this clearly.

Best

Jeff

I understand your question, but that is completely up to you and the performance of the system. If an asset performs spectacularly bad, or if you got bad news about it, remove or replace it. Or if you encounter some promising new assets, add them. The system just determines the optimal portfolio weights. Aside from trading costs there is no particular disadvantage in adding or removing assets.

Hi Johann. Thanks for posting the wonderful articles. I enjoyed reading all of them. The backtest performance of the Z8 strategy is remarkable, with very little drawdowns. I tested the open script you included in this blog using the same 24 ETFs as in the Z8 strategy. But the results are not nearly as good as Z8. You mentioned in an earlier post that Z8 is just a pimped-up version of the script. Would you mind shedding more light on the major differences between the two? Thank you very much!

Z8 is a bit optimized. AFAIK it uses a 150 days lookback period for faster adaption, and a position on the EF very close to minimum variance. Also the weight cap is 4/number of assets.

Thank you for your reply, Johann. After more testing, I found that the Covariance() function in Zoro is defined as Sum((X_i – X_mean)*(Y_i – Y_mean)). Conventionally, covariance is defined as the Sum divided by (LookBack-1). Because the difference is just a scaling matter, it should not affect the backtest performance with the same parameters. But when I tested the code with the modified definition, Cov = Covaraince()/(LookBack-1), the backtest performance looks quite different. Can you please look in it? Thank you very much!

I can not see a difference with the MVOTest script when I multiply the covariance with a factor or divide them by Lookback-1. With which test did you get a difference?

I meant the last script included in your article. I think there may be a small typo in the script. Should the line “var MinVariance = markowitzReturn(0,0);” be “var MinVariance = markowitzVariance(0,0);” instead? Thanks!

Indeed! Thanks for the info. I’ve fixed the wrong line.