Flirting With Models

Sometimes I am working with Rails and just take things for granted. Like before_filter and after_filter. For those who don’t use rails, basically, before_filter and after_filter are called before and after actions are processed. So instead of having to look-up my user from the database in every single action, I can create a before_filter that handles it for me, making the user available in all my actions.

Now what is cool about this is that actions have no special creation system. You just create a new class method, and *bam*, you have a new action for your controller. In that case, what sort of voodoo is going on to prepend and append the filters to our methods? Some sort of ‘alias’ trickery? But that won’t work unless the filters are evaluated after the methods, and that certainly isn’t true for our filters…

Let’s see how deep the rabbit hole goes. Looking at ActionController::Base, we find the perform_action method, which, as you can imagine, performs an action. After extracting the action name and determining whether it is a public action, it tries to ‘send’ the action. Herm, no reference to filters anywhere here.

Next, let’s look at ActionController::Filter. Here, we find a base Filter type, which is a ActiveSupport::Callbacks::Callback. Might as well take a look at that… and all we find is a typical callback object. Well, that got us nowhere. Looking back at the filters source, we can see that when the module is included, it stuffs a whole mess of methods in the base. And that is where the gold is. Look down at the InstanceMethods module. See right there, squeezed in the self.included(base), we find it. The meat and potatoes. alias_method_chain. It seems to be aliasing process and perform_action — the heart of ActionController. But what the hell is alias_method_chain? A bit of poking and prodding and googling, and we find this nice little article which explains to us that alias_method_chain :perform_action, :filters really ends up as alias_method :perform_action_with_no_filters, :perform_action and alias_method :perform_action, :perform_action_with_filters. Likewise for process. Right below, what do we find?

protected
   def process_with_filters(request, response, method = :perform_action, *arguments) #:nodoc:
      @before_filter_chain_aborted = false
      process_without_filters(request, response, method, *arguments)
   end
 
   def perform_action_with_filters
      call_filters(self.class.filter_chain, 0, 0)
   end
 
private
   def call_filters(chain, index, nesting)
      index = run_before_filters(chain, index, nesting)
      aborted = @before_filter_chain_aborted
      perform_action_without_filters unless performed? || aborted
      return index if nesting != 0 || aborted
      run_after_filters(chain, index)
   end

And there we go. A little inheritance and alias magic, and the case is closed.

This gem of an article has been out for quite a while now, and really hits the nail on the head of showing how easy using backgroundrb is.  Or, at least, was.  You see, backgroundrb has gone through some updates and the article is a bit out of date.

So I have spent the last day or two trying to get my locally hosted rails application to talk to a locally hosted backgroundrb client.  After hours and hours of banging my head against the keyboard, gnufied in #backgroundrb (freenet) that I was doing absolutely nothing wrong, but rather it was a known bug with the Mac OS X installation.  Hurrah.

So I decided to test out the whole shebang on EC2 to see if it would work.  And it did.  Now some of you might be wondering how I got such a wonderful little process up and running, so I thought I would share.  Some of you might be able to follow along at home without using EC2.

PLEASE NOTE THAT THESE STEPS ARE FOR TESTING PURPOSES ONLY!  PLEASE ENSURE THAT FOR LIVE DISTRIBUTIONS, YOU EMPLOY CORRECT SECURITY PRACTICES (i.e. don’t be in development mode and put a password on your database, et cetera).

For testing, I chose the basic ruby AMI on EC2.  I then installed rails, chronic, packet, and the mysql gem (–with-mysql-config=/usr/bin/mysql_config).  After, I installed svn (yum install svn).  This gave me all the tools I needed to get up and running.

Next, I created a new rails project (‘rails ec2_client -d mysql’).  You do this because backgroundrb exists within the context of a rails project, even if run in stand-alone.  I navigated my way into vendor/plugins and downloaded backgroundrb (svn co http://svn.devjavu.com/backgroundrb/trunk).  After renaming trunk to backgroundrb (mv trunk backgroundrb), I navigated back to the root of the project directory (cd ../..).

Next, I loaded up mysql and created a development database (‘create database ec2_client_development;’).  After exiting mysql, I then installed the backgroundrb plugin (rake backgroundrb:setup) and migrated the database (rake db:migrate).  Next, I created my worker (script/generate worker some_model).  I opened up the generated worker file (vi lib/workers/some_model_worker.rb) and gave him a name (set_worker_name :some_model.  Please see here for more info).

I then decided to give my worker a little functionality (def ping; cache[job_key] = “pong”; end;).  After, I had to find out my host-name, so I followed the instructions in the tutorial listed above.

wget -q -O /tmp/public-ip http://169.254.169.254/latest/meta-data/public-ipv4
wget -q -O /tmp/public-hostname http://169.254.169.254/latest/meta-data/public-hostname
hostname -F /tmp/public-hostname
echo $(hostname) > /etc/hostname
$(hostname)

Simple enough.  Next, I loaded up my backgroundrb server (script/backgroundrb start -e development -h $(hostname)).  Done and done.  Remember that $(hostname) value!

Before I forget, make sure you have enabled the permissions for your ec2 instances to be allowed to access the port backgroundrb is running in (ec2-authorize default -p 11006).

Now, for your local rails project to chatter with the newly created server is pretty easy.  Here is a sample application I used.

class TestController < ApplicationController
   def index
      host_ip = "ec2-67-202-7-84.compute-1.amazonaws.com"
      port = 11006
      worker = MiddleMan.worker(:some_model)
      result = worker.ping(:host => "#{host_ip}:#{port}", :job_key=> "test")
      render :text => result
   end
end

Nice and simple.  Everything works out gravy.  NB: The ‘host_ip’ here should be the same as the $(hostname) value printed in the EC2 terminal above.

Oh, except it doesn’t.  What is this?  Some sort of error connecting to the server?  What is it doing trying to connect to 0.0.0.0…

You see, when you also installed backgroundrb in your RAILS application (so you could use MiddleMan, remember), it tried to create its own backgroundrb server.  So when the rails application loads, it opens its own backgroundrb config file and takes the server there.  So, a simple alteration to vendor/plugins/backgroundrb/lib/backgroundrb/bdrb_cluster_connection.rb had me comment out ‘establish_connections’ in initialize.

I also wanted my backgroundrb servers to scale dynamically with PoolParty! (as I have mentioned in previous posts).  This was a problem, since backgroundrb only lets me define my servers statically in the configuration file.

Have no fear!  A simple alteration to find_connection in the same file had me off and running:

def find_connection host_info
   conn = @backend_connections.detect { |x| x.server_info == host_info }
   if !conn
      klass = Struct.new(:ip,:port)
      ip = host_info.split(':')[0]
      port = host_info.split(':')[1].to_i
      @bdrb_servers << klass.new(ip,port)
      conn = Connection.new(ip,port,self)
      raise NoServerAvailable.new("BackgrounDRb server is not found running on #{host_info}") unless conn
      @backend_connections << conn
   end
   return conn
end

Now, if it doesn’t find the connection, it creates it (unless it can’t connect, of course).

Now I simply created a Scale controller, which allows backgroundrb servers to register themselves through a simple HTTP::GET (all done at AMI initialization before backgroundrb is started) that logs the host-name.

Now, I haven’t fully figured out how to keep state between MiddleMan’s backend_connections and the database table (whether I actually need the database table yet, I am not 100% sure of) — but getting the ping/pong connection working was a nice little pick-up after several hours of dejection.

Hope someone finds this useful.

I am working on a project that requires some dynamic scaling.  The goal was to have Rails farm out complex processes to Amazon EC2 shards via backgroundrb, and have PoolParty! manage the dynamic scaling.   All good, and what-not, but how was I going to get the new EC2 shards, dynamically loaded by PoolParty!, to get recognized by my web-server?  My first thought was Rinda — specifically, RingyDingy (saving me time on managing connections), but I couldn’t think of a good way of getting the EC2 clients to find the Ring server on my webserver.  Furthermore, how would Rails talk with the Ring server?  Even if I switched out RingyDingy and used Rinda::RingFinger to specify my webserver for the EC2 clients, how would Rails connect?  My first thought was to create a third server that was a backgroundrb server / ring server, and allow Rails to request clients from it.

Then someone on #ruby-lang said I should just have the EC2 clients register and de-register themselves via HTTP GET, and just manage them via a table in my database.

As always, K.I.S.S.