Recently in rubyonrails Category

Complex SQL Queries with Rails

February 18, 2007 1:39 PM

Rails is a powerful framework - it abstracts many of the underlying database concepts so that you never have to think about them. Of course, sometimes it abstracts away too many.

Frequently I run into a case where I want more data in an ActiveRecord than is actually available from my SQL schema. For instance, I often want to include data from another table through a JOIN or sub-SELECT, or maybe I want to be able to ORDER BY data in a table that's in a belongs_to relationship with the current model.

To illustrate why this is useful, let's assume that we have a table, Pet, which has a PetType. Our schema looks something like this:

pet_types Table:

idname
1Dog
2Platypus
3Cat

pets Table:

idnamepet_type_id
1Santa's Little Helper1
2Snowball3
3Brian1
4Fritz3
5Felix3

Given this schema, there are two things I want to be able to do:

  1. Get a list of pets, ordered by their type. But I want to order on pet_types.name, not pet_types.id. (That is, I want them in order of "Cats", "Dogs", "Platypi", not whatever order their ids are.)
  2. Get a list of pet types, including a count of the number of pets which are of that type.

Admittedly, in this contrived example, I could do these both in rails without making any changes to the model and without too much pain. But with more advanced queries, it's much easier to let your database engine do the work. And rails is relatively graceful about allowing you to do this -- you do need to know a little SQL, but you only need to change a single method in your model.

Ordering by a foreign column
Let's say you wanted to display a list of pets ordered by type. We'd like our database engine to do this for us, since this is one of the things it's optimized at. Since ActiveRecord::Base's various find_* methods all ultimately call find_every(), we need only override that method to add a more complex SQL query. We just need to make one change in our model to support this:

In models/pet.rb:

def Pet.find_every(args = { })
  sql = "SELECT pets.*, pet_types.name AS pet_type_name " +
      "FROM pets, pet_types " +
      "WHERE pets.pet_type_id = pet_types.ID"

  sql += " AND " + sanitize_sql(args[:conditions])
      if(args[:conditions])
  sql += " ORDER BY " + sanitize_sql(args[:order])
      if(args[:order])
  sql += " LIMIT " + sanitize_sql(args[:limit].to_s)
      if(args[:limit])
  sql += " OFFSET " + sanitize_sql(args[:offset].to_s)
      if(args[:offset])

  find_by_sql(sql)
end

Now we can say Pets.find(:all, :order => 'pet_types.name') to sort by pet type. This will give us the following:

idnamepet_type_idpet_type_name
2Snowball1Cat
4Fritz1Cat
5Felix1Cat
1Santa's Little Helper1Dog
3Brian1Dog

Including data from another table
Let's say that we wanted to provide a list of pet types and inc lude the number of pets of that type. We could certainly load each pet type, then do a Pet.count() for each type, but it may be advantageous to let the database engine do this for us.

In models/pet_type.rb:

def PetTypes.find_every(args = { })
  sql = "SELECT pet_types.*, " +
      "(SELECT COUNT(pets.id) FROM pets " +
      "WHERE pets.pet_type_id = pet_type.id) AS pet_type_count " +
      "FROM pet_types "

  sql += " AND " + sanitize_sql(args[:conditions])
      if(args[:conditions])
  sql += " ORDER BY " + sanitize_sql(args[:order])
      if(args[:order])
  sql += " LIMIT " + sanitize_sql(args[:limit].to_s)
      if(args[:limit])
  sql += " OFFSET " + sanitize_sql(args[:offset].to_s)
      if(args[:offset])

  find_by_sql(sql)
end

Now, we get a new column - pet_type_count which we can read (but not write to, of course) as if it were part of the table. We can use it just for display, or we can sort on it just like above. PetType.find(:all, :order => 'pet_type_count') will produce:

idnamepet_type_count
2Platypus0
1Dog2
3Cat3

While these are trivial examples, this sort of overriding can become exceptionally useful when you have a lot of data being computed by the SQL server. If you have a model with several belongs_to or has_many relationships, it's a lot easier to let the SQL server compute these sort of details itself, rather than trying to do it in each method of your model.

Ultimately, you can put any arbitrary SQL into your find_every() method, and rails will continue to work as usual, provided you follow these three simple rules:

  1. Don't exclude columns, only include new ones. Rails examines your table's columns the first time you use a model, and it will expect those columns to be in the results of any query.
  2. Fully qualify conflicting column names. That is, say: :order => 'pets.name' instead of just 'name'.
  3. Ensure that you handle the arguments to find_every(). That is, remember to handle :conditions, :order, :limit and :offset. Otherwise you run the risk of breaking other finds.

Teamprise for TextMate

May 25, 2006 11:50 AM

TextMate Teamprise and Microsoft Team Foundation Server users can now access their source code control directly from the TextMate editor.

TextMate is a popular developer-oriented text editor for Mac OS X. It's particularly popular in the interpreted language camps due to its light weight, high configurability and out-of-the-box support for many languages. Rightly so - it's a great programming text editor, and I've been using it anytime I don't need the full featureset of an IDE like Eclipse.

My only complaint with TextMate was that I constantly had to flip between my editor window and Teamprise Explorer so that I could check out and check in files.

TextMate ships with support for source control repositories in the form of Subversion and CVS. This is nice - but I need to check in to a TFS server. No problem! Thanks to TextMate's configurable architecture, I was able to roll my own interface to Teamprise in the form of a command "bundle."

The TextMate support for Teamprise went surprisingly smoothly. Using the Subversion and CVS bundles as a guide, I had a working prototype using the Teamprise command line client in just a bit over an hour. After that, I went back in to add a few features and polish the checkin functionality using a helpful utility included in the Subversion bundle. Now major features are supported -- add, delete, check out, check in, view history, undo changes, etc.

You can download this bundle at http://people.teamprise.com/~ethomson/teamprise/textmate/.

I'll admit, it's no stellar piece of engineering, and there are surely bugs. This was one of those little utilities born of necessity. That said, I hope the other Mac-based TFS users find some utility with it.

Of course, you will need a copy of the Teamprise Command Line Client and a Teamprise client license. This is not officially supported by Teamprise.

I've been playing with Ruby on Rails for a few days now, slowly grokking Ruby's unique syntax, sparsely documented stdlib[1] and lack of semicolons. The first stumbling block, albeit simple, was Rails' seemingly simple error_messages_for.

In a view, <%= error_messages_for 'MODEL' %> will show you any errors from MODEL's validate method. It's straightforward enough... unless you're trying to validate against a second model as well, from a validate_associated. Then MODEL's only error message is "othermodel is invalid". Not terribly helpful. You could draw a second box explaning why, but that's only slightly more enlightening (if at all.)

It would be ideal if we could just put all the error messages in a single error block. So what we need to do is get rid of the "othermodel is invalid" error, and add the other model's errors. For example:

errors = ActiveRecord::Errors.new(nil) @model.errors.each { |key, message| errors.add(key, message) unless key == "othermodel" } @othermodel.errors.each { |key, message| errors.add(key, message) } @model.errors.clear errors.each { |key, message| @model.errors.add(key, message) }

Sort of a hack, I'll admit. But hey, it works, and we can do this at any point after valid?[2] has been called on your models. (valid? actually populates @model.errors.)

Here's a detailed example -- let's say you have two objects, a Transaction and an Account, and you want a view that simply asks for an amount for a transaction and an account number, and you want to show the user if they enter an invalid amount or an invalid account number. Your models and views are going to look something like this:

views/transaction/create.rhtml:

<%= error_messages_for 'transaction' %>
<%= start_form_tag :action => 'create' %>
<label for="transaction_amount">Amount:</label> <%= text_field :transaction, :amount %>
<label for="account_number">Account:</label> <%= text_field :account, :number %>
<%= end_form_tag %>

models/transaction.rb:

class Transaction < ActiveRecord::Base
  belongs_to :account
  validates_presence_of :amount, :message =< "is a required field"
  validates_associated :account
end

models/account.rb:

class Account < ActiveRecord::Base
  has_many :transactions
  validates_presence_of :number
end

controllers/transaction_controller.rb:

class TransactionsController < ApplicationController
  def create
    if request.post?
      @account = Account.find(:first, :conditions => [ "number = ?", @params[:account][:number]])
      @transaction = Transaction.new()
      @transaction.account = @account
      @transaction.amount = @params[:transaction][:amount]
      if @transaction.save
        redirect_to :action => 'show', :id => @transaction
      else
        # here's where we get the good errors
        errors = ActiveRecord::Errors.new(nil)
        @transaction.errors.each { |k,m| errors.add(k,m) unless k == "account" }
        @account.errors.each { |k,m| errors.add(k,m) }
        @transaction.errors.clear
        errors.each { |k,m| @transaction.errors.add(k, m) }
      end
    end
  end
end

Now, if you were to leave both fields blank, you'd get an error message like:

  • Amount is a required field
  • Number can't be blank

Which is the error message that one would expect out of this view.[3]

  1. Really, the stdlib documentation is almost nonexistant! Digest? OpenSSL? Hah! Your best bet is to look at code samples and guess!
  2. valid? is called by save, for the record
  3. Far superior to the default:
    • Amount is a required field
    • Account is invalid
Edward Thomson is a Software Engineer at Teamprise, where he develops cross-platform client solutions for Microsoft Team Foundation Server, with an emphasis on Macintosh compatibility and IDE integration.