Flirting With Models

This gem of an article has been out for quite a while now, and really hits the nail on the head of showing how easy using backgroundrb is.  Or, at least, was.  You see, backgroundrb has gone through some updates and the article is a bit out of date.

So I have spent the last day or two trying to get my locally hosted rails application to talk to a locally hosted backgroundrb client.  After hours and hours of banging my head against the keyboard, gnufied in #backgroundrb (freenet) that I was doing absolutely nothing wrong, but rather it was a known bug with the Mac OS X installation.  Hurrah.

So I decided to test out the whole shebang on EC2 to see if it would work.  And it did.  Now some of you might be wondering how I got such a wonderful little process up and running, so I thought I would share.  Some of you might be able to follow along at home without using EC2.

PLEASE NOTE THAT THESE STEPS ARE FOR TESTING PURPOSES ONLY!  PLEASE ENSURE THAT FOR LIVE DISTRIBUTIONS, YOU EMPLOY CORRECT SECURITY PRACTICES (i.e. don’t be in development mode and put a password on your database, et cetera).

For testing, I chose the basic ruby AMI on EC2.  I then installed rails, chronic, packet, and the mysql gem (–with-mysql-config=/usr/bin/mysql_config).  After, I installed svn (yum install svn).  This gave me all the tools I needed to get up and running.

Next, I created a new rails project (‘rails ec2_client -d mysql’).  You do this because backgroundrb exists within the context of a rails project, even if run in stand-alone.  I navigated my way into vendor/plugins and downloaded backgroundrb (svn co http://svn.devjavu.com/backgroundrb/trunk).  After renaming trunk to backgroundrb (mv trunk backgroundrb), I navigated back to the root of the project directory (cd ../..).

Next, I loaded up mysql and created a development database (‘create database ec2_client_development;’).  After exiting mysql, I then installed the backgroundrb plugin (rake backgroundrb:setup) and migrated the database (rake db:migrate).  Next, I created my worker (script/generate worker some_model).  I opened up the generated worker file (vi lib/workers/some_model_worker.rb) and gave him a name (set_worker_name :some_model.  Please see here for more info).

I then decided to give my worker a little functionality (def ping; cache[job_key] = “pong”; end;).  After, I had to find out my host-name, so I followed the instructions in the tutorial listed above.

wget -q -O /tmp/public-ip http://169.254.169.254/latest/meta-data/public-ipv4
wget -q -O /tmp/public-hostname http://169.254.169.254/latest/meta-data/public-hostname
hostname -F /tmp/public-hostname
echo $(hostname) > /etc/hostname
$(hostname)

Simple enough.  Next, I loaded up my backgroundrb server (script/backgroundrb start -e development -h $(hostname)).  Done and done.  Remember that $(hostname) value!

Before I forget, make sure you have enabled the permissions for your ec2 instances to be allowed to access the port backgroundrb is running in (ec2-authorize default -p 11006).

Now, for your local rails project to chatter with the newly created server is pretty easy.  Here is a sample application I used.

class TestController < ApplicationController
   def index
      host_ip = "ec2-67-202-7-84.compute-1.amazonaws.com"
      port = 11006
      worker = MiddleMan.worker(:some_model)
      result = worker.ping(:host => "#{host_ip}:#{port}", :job_key=> "test")
      render :text => result
   end
end

Nice and simple.  Everything works out gravy.  NB: The ‘host_ip’ here should be the same as the $(hostname) value printed in the EC2 terminal above.

Oh, except it doesn’t.  What is this?  Some sort of error connecting to the server?  What is it doing trying to connect to 0.0.0.0…

You see, when you also installed backgroundrb in your RAILS application (so you could use MiddleMan, remember), it tried to create its own backgroundrb server.  So when the rails application loads, it opens its own backgroundrb config file and takes the server there.  So, a simple alteration to vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_cluster_connection.rb had me comment out ‘establish_connections’ in initialize.

I also wanted my backgroundrb servers to scale dynamically with PoolParty! (as I have mentioned in previous posts).  This was a problem, since backgroundrb only lets me define my servers statically in the configuration file.

Have no fear!  A simple alteration to find_connection in the same file had me off and running:

def find_connection host_info
   conn = @backend_connections.detect { |x| x.server_info == host_info }
   if !conn
      klass = Struct.new(:ip,:port)
      ip = host_info.split(':')[0]
      port = host_info.split(':')[1].to_i
      @bdrb_servers << klass.new(ip,port)
      conn = Connection.new(ip,port,self)
      raise NoServerAvailable.new("BackgrounDRb server is not found running on #{host_info}") unless conn
      @backend_connections << conn
   end
   return conn
end

Now, if it doesn’t find the connection, it creates it (unless it can’t connect, of course).

Now I simply created a Scale controller, which allows backgroundrb servers to register themselves through a simple HTTP::GET (all done at AMI initialization before backgroundrb is started) that logs the host-name.

Now, I haven’t fully figured out how to keep state between MiddleMan’s backend_connections and the database table (whether I actually need the database table yet, I am not 100% sure of) — but getting the ping/pong connection working was a nice little pick-up after several hours of dejection.

Hope someone finds this useful.