Sep 2 2009

As it turns out, you may be a unique snowflake after all…

Max Dama puts up some interesting notes on the “Curse of Dimensionality”.

  • Share/Bookmark

Aug 31 2009

Don’t forget the second derivative…

Turns out, the “flying wing” design of the B-2 bomber was a range minimizer, not a maximizer. Someone forgot to check that second derivative!

Read more here.

  • Share/Bookmark

Aug 31 2009

A eulogy to _why

Where, oh where, has _why gone?  A beautiful ruby ‘artist.’  For many, he brought the magic back to programming — the joy that had been sucked out in the tech boom when programmers became a commodity.  John Resig has written this fitting eulogy to _why.  The comments are great too.

_why, I don’t know where you are, but thanks for all you’ve done.

  • Share/Bookmark

Aug 19 2009

HotSwapping in languages that aren’t Erlang

Here is a great article on how to HotSwap Code using Scala and Actors.  It brings up the question, how can hotswapping be implemented in languages where hotswapping was not necessarily intended?  Can it really be as simple as creating a wrapper module that swaps the instances?  If you can somehow make guarantees about state (once again, functional programming to the rescue!), I don’t see why not!

  • Share/Bookmark

Jul 26 2009

Measuring Your Model’s Performance

In my last post, I wrote about a simple method to determine if your model is statistically significant by seeing if it “out-performs” random. There are a lot of ways to measure your model’s performance. They start as simple as measuring return, and only get more complex from there. After my last post, a friend of mine wrote me an e-mail about a method he uses to test the performance of his model that I thought was interesting and would pass along.

First, we determine what the model is trying to capture. What we would then do is create a ‘perfect’ system, by looking ahead into the future. We then run this perfect system on data to generate what he would consider ‘perfect’ signals.

We can then create fitness functions to measure your systems performance against the theoretical perfect performance. We might, for example, use a ‘time matching’ measurement, where we measure how closely our system’s trades match those of the perfect system. If our model has the same signals for 95% of the time, we know we are probably doing something right.

The power of this system lies in creating accurate and telling fitness functions. Preferably, your fitness functions should give some way to measure statistical confidence levels.

Cool concept. One to throw in the bag.

  • Share/Bookmark

Jun 16 2009

Free Matlab Econometrics Toolbox

Here is a great, free econometrics tool-box, courtesy of James P. LeSage. It has an accompanying manual which can be downloaded here. Just make sure you add the library location to your Matlab search path (File > Set Path)!

Also on the site is a long (200 pages) book (?) on Spatial Econometrics, which you can get here. As well, he has another long (300 pages) book on The Theory and Practice of Spatial Econometrics. Grab it here.

And finally, also available is his book Applied Econometrics using MATLAB. 350 pages, and definitely worth a look!

  • Share/Bookmark

Jun 16 2009

Sorting with Ruby, RubyInline, and Thrust

I decided to see what sort of speed-up I could get just doing a straight brute-force sort. First, I wanted a pure Ruby implementation. Next, I wanted a RubyInline version. Finally, I wanted to use RubyInline with Thrust. Here is the code I used.

Please NOTE: I understand that the appropriate choice of an algorithm is more important than ‘micro’ optimizations. I just thought it would be fun to check out.

# Ruby Sort Neighbors example
# 	Take N random (x,y) points
#	For each point, sort the neighbors!
 
require 'rubygems'
require 'inline'
 
N = 1_000
 
class Point
	attr_accessor :x, :y
 
	def initialize(x, y)
		@x = x
		@y = y
	end
 
	def distance(other)
		Math.sqrt((other.x - @x)**2 + (other.y - @y)**2)
	end
end
 
def find_nearest_neighbors(point, neighbors)
	return neighbors.sort { |a,b| point.distance(a) <=> point.distance(b) }
end
 
class NearestNeighbor
 	inline(:C) do |builder|
	    builder.add_compile_flags '-x c++', '-lstdc++', '-I/usr/local/include/boost-1_39/'
		builder.include '<algorithm>'
		builder.include '<vector>'
	    builder.include '<boost/bind.hpp>'
		builder.include '<cmath>'
 
		builder.prefix '
			int distance(VALUE a, VALUE b, VALUE c) {
		      	double a_x = NUM2DBL(rb_iv_get(a, "@x"));
			    double a_y = NUM2DBL(rb_iv_get(a, "@y"));
			    double b_x = NUM2DBL(rb_iv_get(b, "@x"));
			    double b_y = NUM2DBL(rb_iv_get(b, "@y"));
			    double c_x = NUM2DBL(rb_iv_get(c, "@x"));
			    double c_y = NUM2DBL(rb_iv_get(c, "@y"));
 
			    double b_dist = sqrt(pow(b_x-a_x, 2) + pow(b_y-a_y, 2));
			    double c_dist = sqrt(pow(c_x-a_x, 2) + pow(c_y-a_y, 2));
 
			    return (b_dist < c_dist);
      		}'
 
	    builder.c '
	    	VALUE find_nearest_neighbors(VALUE point, VALUE neighbors) {
	      		const VALUE *base = RARRAY(neighbors)->ptr;
	 			std::vector<VALUE> sorted_neighbors(base, base + RARRAY(neighbors)->len);
 
	      		std::sort(sorted_neighbors.begin(), sorted_neighbors.end(), boost::bind(&distance, point, _1, _2));
 
	      		return rb_ary_new4(RARRAY(neighbors)->len, &sorted_neighbors.front());
	      }'
  	end
end
 
class NearestNeighborThrust
 	inline(:C) do |builder|
	    builder.add_compile_flags '-x c++', '-lstdc++', '-I/usr/local/cuda/include'
		builder.include '<vector>'
		builder.include '<cmath>'
 
		builder.include '<thrust/sort.h>'
 
		builder.prefix '
			float distance(VALUE a, VALUE b) {
		      	double a_x = NUM2DBL(rb_iv_get(a, "@x"));
			    double a_y = NUM2DBL(rb_iv_get(a, "@y"));
			    double b_x = NUM2DBL(rb_iv_get(b, "@x"));
			    double b_y = NUM2DBL(rb_iv_get(b, "@y"));
 
			    return sqrt(pow(b_x-a_x, 2) + pow(b_y-a_y, 2));
      		}'
 
	    builder.c '
	    	VALUE find_nearest_neighbors(VALUE point, VALUE neighbors) {
	      		VALUE *base = RARRAY(neighbors)->ptr;
	      		const int length = RARRAY(neighbors)->len;
 
	 			std::vector<double> keys(length);
 
	 			for(int i = 0; i < length; i++)
	 				keys[i] = distance(point, base[i]);
 
	      		thrust::sort_by_key(&keys.front(), &keys.front()+length, base);
 
	      		return rb_ary_new4(length, base);
	      }'
  	end
end
 
# initialize our points
points = []
N.times {
	points << Point.new(rand, rand)
}
 
nearest_neighbors = []
start = Time.now
points.each_index { |i|
	nearest_neighbors[i] = find_nearest_neighbors(points[i], points[0...i] + points[i+1..(N-1)])
}
finish = Time.now
puts "Ruby Time: #{finish-start}"
 
nearest_neighbors_inline = []
nn = NearestNeighbor.new()
start = Time.now
points.each_index { |i|
	nearest_neighbors_inline[i] = nn.find_nearest_neighbors(points[i], points[0...i] + points[i+1..(N-1)])
}
finish = Time.now
puts "Inline Time: #{finish-start}"
 
nearest_neighbors_thrust = []
nn_thrust = NearestNeighborThrust.new()
start = Time.now
points.each_index { |i|
	nearest_neighbors_thrust[i] = nn_thrust.find_nearest_neighbors(points[i], points[0...i] + points[i+1..(N-1)])
}
finish = Time.now
puts "Inline w/ Thrust: #{finish-start}"

Here are the results I got on my new MacBook 13″:
Ruby Time: 46.998292
Inline Time: 5.911666
Inline w/ Thrust: 0.726583

  • Share/Bookmark

May 29 2009

Some Notes on Neural Networks

What are hidden layers and how many should I use?

Some general Q&A about neural networks

  • Share/Bookmark

Feb 15 2009

Poignant

6a00d83451587d69e2010537203ca8970b-800wi

  • Share/Bookmark