Flirting With Models

There is quite a bit written about the 6-1 momentum model, which ranks investments by their previous 6 month return, minus the previous 1 month return, and holds for 1 month.

The idea behind the method is that average returns over the long run should be stable in the short run, made superior by taking into account short-term reversionary factors.

To explore this concept and see exactly how stable the 6-1 model is, I explored 30 diverse ETFs (equities, commodities, currencies, bonds, et cetera) over a period of 400 days, changing my momentum, reversion, and prediction periods.

To begin, I transformed adjusted closing prices into cumulative log returns.
I calculate the prediction returns as follows (in matlab syntax):

pastReturns = (cumulativeReturns(today, :) - cumulativeReturns(today-n, :)) - (cumulativeReturns(today, :) - cumulativeReturns(today-m, :));

and the future returns as

futureReturns = (cumulativeReturns(today+p, :) - cumulativeReturns(today, :));

I then sort both sets of returns and calculate the difference between their sorted order to get the total estimation error

[v, idx] = sort(returns);
[v, futureIdx] = sort(futureReturns);
error = sum(abs(idx - futureIdx));

I do this, looping through n, m, and p. To generate the total error for a given (n,m,p) tuple, I loop through all available days, compute the prediction error for that day, and average the errors.

The results are visualized as follows (please note, axes must be scaled by a factor of 5 and that the data base been interpolated to fill in missing spots):

To me, there are two very intriguing things about this visualization.

First, there seems to be a region of stability (or am I just seeing what I want to see?) in the 90-110 day momentum, 25-50 day reversion, and 20-30 day prediction area, indicating that there may be some validity to the 6-1 momentum method. However, it is not the ONLY region of stability, bringing into question its validity.

Secondly, the line where the momentum look-back period equals the reversion look-back period (i.e. when there is NO prediction data and all stocks are expected to perform equally well), we get the minimum error. However, to keep a consisten equal balance in the portfolio would incur incredibly high turn-over, and I suspect that the benefit of this error decrease would actually be more costly due to trading costs & slippage.

So while 6-1 may out-perform buy and hold, it seems that a constantly equally rebalancing portfolio may be worth exploring further.

Back testing is fairly common when analyzing the profitability of a strategy, but there are many other things to be considered besides returns.  Much of this list came from perusing Nuclear Phynance (particularly, FDAXHunter’s input).

• Length of the Period Tested: Over what time-frame did you run the test? How long was that time-frame?  What market conditions persisted over it?  The goal should be to run the strategy on as diverse a time-frame as possible, to help you discover what market factors play a critical role in the success or failure of your method.
• Out of Sample Tests & Locations: Much like above, you want to have a fairly significant and diverse set of out of sample data to test on.
• Average Trade / Win / Loss: What does the average trade of the system look like?  Are you normally profitable, or do you lost money and it was a couple fat-tail trades that gave you profitability?
• Volatility & Skewness of P&L Stream: Is your profitability stable?  Do you have a fat loss tail?  How skewed positive are you?
• Maximum Consecutive Losers / Winners: This is very important for when the system goes live.  Is five bad trades in a row a reason to pull the system?  Ten?  What is normal for the system?  When should we start getting concerned?
• Maximum Draw Down & Time: If we implemented the system, what sort of draw-downs would we have to stomach, and over how long would we have to stomach them?  I don’t care much about a 1300% return over 5 years if for 4 of them, I faced an 80% draw-down.  You would probably pull the plug long before the fifth year came around.
• Average Draw Down & Time: What does the average draw down look like?  Is it stable?
• Percent of Winners Removed Until Neutral: What percentage of our best trades do we have to remove before the system breaks?  Is it only a few?  Is our success based on a few large winners, or do we have a stable set of success?
• Histogram of P&L: What does the P&L look like, historically?  This is a visualization of the skew and volatility from above.
• Shape of Equity Curve: Are we talking about a long, smooth curve?  A curve with lots of jumps?  How much interim volatility between new highs?
• Optimal Parameter Location: Are the parameters we used in a stable location, or are they at a pin-point?  If they are at a pin-point, the success of the model is most likely the result of data-mining, instead of a true edge.  Instead, we would like to see that our parameters are in a plateau — the model remains stable for moderate changes in our parameter values.
• Performance in Other Markets: How does the model perform on securities it wasn’t designed to trade for?  Does it succeed in similar securities?  Does it fail in securities which the edge shouldn’t exist on?

All of these things should be considered along-side a simple profitability analysis, or else you will end up with a ‘successful back-test of three years, blow up in three days’ scenario.

Always on the look-out for interesting system measurements, I stumbled across the “e-ratio,” described here. The concept is to graph your normalized (volatility-adjusted) MFE (maximum favorable excursion) to MAE (maximum adverse excursion) ratio for different n (time-steps) since a trade was taken. A value above 1 allows you to identify a positive edge. It is most useful when making tweaks to a system, to help identify when a filter or signal adds ‘edge’ over all time-steps.

Random programming libraries that I have discovered in trying to solve some problems lately…

C++
Acedia: acedia is designed to be an easy to use C++ library. It provides an ERLANG like actor implementation with message passing and pattern matching. It’s main goal is to support development of distributed software – also across a network.
POCO: The POCO C++ Libraries (POCO stands for POrtable COmponents) are open source C++ class libraries that simplify and accelerate the development of network-centric, portable applications in C++.

Ruby
Sinatra: Embeddable website DSL
Nanite: Nanite is a new way of thinking about building cloud ready web applications. Having a scalable message queueing back-end with all the discovery and dynamic load based dispatch that Nanite has is a very scalable way to construct web application back-ends.
Journeta: Journeta is a dirt simple library for peer discovery and message passing between Ruby applications on a LAN.

That’s all for now. I’ve been working on some interesting stuff, developing a client registration and authentication system. I started out in C++ using boost::ASIO to handle server connections, but then I just got annoyed, switched to Ruby, and have been running along ever since. I am using Revactor (with a few personal tweaks in the code) to handle my components, SQLite3 for a simple database (though, I might have to switch to MySQL for production), and Sinatra for a web-interface (running on Thin with Shotgun). All in all, I must say that simply plugging in working components instead of having to develop the resources makes production so much easier… As always, hardware is cheaper than programmer hours.

Speaking of which, why did I not notice Amazon’s Auto Scaling and Load Balancing features before? If you don’t have the hardware resources to build your own cloud (Eucalyptus with Xen), Amazon really does seem like a very cheap alternative. I mean, think of all the money you save not having to hire I.T. guys to figure out why your harddrive failed in the middle of the night. Amazon does it all for you!

Hopefully, some sort of Cloud Computing standard will come out soon. The only current problem with Amazon’s model is that it doesn’t give the clients any leverage. Start using Amazon and you can’t make the threat to leave and go to Google (well, easily at least). Worse yet, if Amazon goes out of business, you are in a world of hurt. Eucalyptus adopted the Amazon scheme, so at least there is a bit of leverage saying that you will just buy your own hardware — but it would be much better if we could get some pricing competition going on.

Just launched a new page called “Automated Trading Resources.” It is a compiled list of all the trading resources I have gathered over the last several years. From discussions to applications to language specific libraries — it has a whole lot there. Check it out. You can find the link in the navigation bar above.