Posted on Wednesday, August 26th, 2009 at 8:45pm by Rob Wilkerson
Whenever I decide to pick up a new language – or framework – there are usually two ubiquitous problems that I like to try to solve immediately: authentication and file uploads. Over the years I’ve found that these are great learning tools for a couple of reasons.
- They’re well known problems. They’ve been around for a long time and the best practices for solving them are pretty well documented.
- They’re finite problems. There are only so many ways to skin these particular cats and the solutions are very contained. I could build an app whose sole purpose is to login, upload a file and display it.
- The problems may be well known and finite, but they’re not trivial. Trivial solutions don’t make for good learning tools.
This month, I’ve finally taken the plunge and picked up Ruby and Rails. That’s a two-fer (a language and a framework), so once again I started figuring out how to solve these same problems with the new toolkit.
Authentication
After looking around, I found that there is already an authentication plugin, AuthLogic, that looked remarkably comprehensive and solved that particular problem just about the same way I would have if I’d done it manually, but then threw a few more useful tools in the kit that I probably wouldn’t have. Additionally, it looked relatively easy to install and configure.
That sounded like a pretty good deal to me, so I followed along with Ryan Bates’ excellent Railscast and got authentication hooked up rather quickly. No point in re-inventing the wheel as long as the existing wheel is as good as or better than the one you’d have invented, right?
Upload
I did not, however, find a similarly comprehensive plugin for file uploads. There are a few out there – Attachment_fu and Paperclip seem to lead the pack – but none that appear to handle physical media the way I like to handle it. Since my wheel is better than (or at least different from) those wheels, this became my de facto learning experience.
Disclaimer: I only took a quick look at what these plugins had to offer, so maybe they’re closer to my way than it appeared. Nonetheless, I needed a problem to solve and this became the one.
Requirements
As I mentioned, my requirements are slightly different from most I’ve seen, so it’s probably worth outlining them really quickly.
- Store physical files to their own directory structure dedicated to user contributed content. In my Rails project, I have a Rails.root/public/bin/ directory for this. However, since I don’t want to store (what I hope will be) thousands of files or more in a single directory, I create 26 subdirectories named a-z. In each of those, I create 26 more directories also named a-z. Now I have 676 possible locations so my (potentially) huge number of files can be nicely distributed (for example, Rails.root/public/bin/r/m/file.ext) for performance.
- Store physical file metadata (file size, MIME type, URI, etc.) in the database. I often find myself needing or wanting to query this kind of data directly and I don’t want to have to do so through the application. I want easy access to find out how many images I have that are over 50KB, for example.
- Abstract the common functionality to allow the upload of many different types of file (images, videos, flash movies, etc.) with very little effort. In a traditional inheritance model, I might have a File class (a concrete class) that is extended by any type with specialized properties or actions.
For the sake of this article, I’ll keep this simple and focus on images as the only subtype. The extrapolation should be easy enough.
Approach
As I alluded to above, an image is-a file, so a simple inheritance model would be a perfect fit, but…
- Rails doesn’t offer much in the way of inheritance (no, single table inheritance is not a sufficient solution).
- “File” is a reserved word in the Rails framework.
The latter problem was easy to solve, so I created an Image model and a Binary model. The first, though, was a little trickier. Although Rails’, well, ActiveRecord’s, support for inheritance is non-existent, Rails offers something very helpful for this purpose: observer support.
At a Really High Level
From 10,000’, it’s quite simple (isn’t everything at that distance?). I create an image, but before the image object is actually saved, I want to hand off the physical file to the Binary class for uploading to a temporary directory, inspection and extraction of metadata. If that’s successful, I want to move the file to a permanent location and continue saving the image. If anything fails at any point in the process, I want to remove any database records and delete the physical file.
Code
Enough with the chit-chat, let’s write some code.
View
The only relevant view code is that for the form partial. It allows the user to enter a title for the image, a description and to select a file. Images have other properties (like width and height), but they’re derived and I don’t want the user to enter those.
<% form_for( @image, :html => { :multipart => true } ) do |f| %>
<%= f.error_messages -%>
<p>
<%= f.label :name, 'Title' %>
<%= f.text_field :name %>
</p>
<p>
<%= f.label :upload, 'File' -%>
<%= f.file_field :upload -%>
</p>
<p>
<%= f.label :description %>
<%= f.text_area :description %>
</p>
<p><%= f.submit( 'Upload' ) %></p>
<% end %>
Controller
As with most, there are two controller actions in play to upload a new image: new and create.
class ImagesController < ApplicationController
def new
@image = Image.new
end
def create
@image = Image.new( params[:image] )
@image.user_id = current_user
@image.save!
flash[:notice] = “Successfully created image.”
redirect_to @image
rescue => e
logger.error( ‘Upload failed. ‘ + e )
flash[:error] = ‘Upload failed. Please try again.’
render :action => ‘new’
end
end
The only significant – and perhaps non-obvious – difference from the basic scaffold code is the move away from if @image.save! to an exception-centric approach. I did that, first, because I can raise exceptions anywhere and how they’re handled is absolutely predictable. I can’t always return false and be sure that activity simply stops. Second, Rails has a very nice feature, in my opinion, that encapsulates the entirety of a save action in a transaction. If an exception is raised at any time during the save process, the transaction is rolled back and it’s like nothing ever happened. That’s a lot of goodness for absolutely no effort and I wanted to take advantage of it.
Models and Observer
I find it easiest to think about the model code in the order in which it’s encountered, so I’m going to split the code views and build on them accordingly. If that proves unpopular, I’ll publish a more unified version of the model code.
First, a snippet of the Image model to show its association, accessor attribute and validation.
class Image < ActiveRecord::Base
belongs_to :binary
validates_presence_of( :upload )
attr_accessor :upload
# snip (for now)
end
The observer is watching the Image model. When the save process is kicked off (and because we’re creating a new image), the BinaryObserver is engaged and its before_create callback is executed. In that callback we’re offering the Image model the opportunity to execute any instructions it may have before_ the physical file is actually uploaded. If no exceptions are raised, the Binary model is told to upload the file.
class BinaryObserver < ActiveRecord::Observer
observe :image
def before_create( model ) # in this case, the Image model is passed
if model.respond_to?( ‘before_upload’ )
model.before_upload( model ) rescue raise
end
binary = Binary.new.upload( model.upload )
# snip (for now)
end
The file uploaded by the form is passed to the Binary model and stored to a temporary location on the server. The upload method stores the file in a temporary location so that it can be further inspected and/or validated by the subtype for any details that may be relevant to that subtype. Any file that remains in the temporary location is assumed to be orphaned and is subject to deletion. Any file that makes it into the bin/ directory hash is assumed to be live and is left alone unless there’s a good reason for touching it.
class Binary < ActiveRecord::Base
has_one :image
def upload( uploaded_file )
self.name = uploaded_file.original_path
self.mime_type = uploaded_file.content_type
# get_bin_root() returns File.join( Rails.root, ‘public’, ‘bin’ )
save_as = File.join( get_bin_root(), ‘_tmp’, uploaded_file.original_path )
File.open( save_as.to_s, ‘w’ ) do |file|
file.write( uploaded_file.read )
end
self.extension = File.extname( self.name ).sub( /^\./, ‘’ ).downcase
self.size = File.size( save_as )
self.path = save_as.sub( Rails.root.to_s + ‘/’, ‘’ )
self.uri = get_uri_from_path()
self.save!
return self
end
end
Since this is a specific type of file – an image, I want to extract certain additional properties (in this case, the image’s width and height) from the physical file before the final save. I may also want to perform additional validation on the physical file while it’s stored in its temporary location on the file system. That’s where the custom after_upload callback comes into play.
# models/binary_observer.rb
class BinaryObserver < ActiveRecord::Observer
observe :image
def before_create( model )
if model.respond_to?( ‘before_upload’ )
model.before_upload( model ) rescue raise
end
binary = Binary.new.upload( model.upload )
if model.respond_to?( ‘after_upload’ )
model.after_upload( model, binary ) rescue raise
end
end
The original model (Image, in case you’ve forgotten) is passed along with the Binary object. The image is read and additional metadata pertinent only to images (not to generic files) is extracted and written to the model.
# models/image.rb
require ‘RMagick’ # The rmagick gem is required to inspect/manipulate images
class Image < ActiveRecord::Base
belongs_to :binary
validates_presence_of( :upload )
attr_accessor :upload
def after_upload( model, file )
# Insert any physical file validation requirements here
image = Magick::Image::read( file.path ).first
self.width = image.columns
self.height = image.rows
end
end
Assuming everything goes well, we now have complete Binary and Image models as well as a valid file on our file system. To close out, we’re going to tell the Binary class to move the file to its permanent location, update any model properties accordingly and send the user to the image page so they can see their new upload.
class BinaryObserver < ActiveRecord::Observer
observe :image
def before_create( model )
if model.respond_to?( ‘before_upload’ )
model.before_upload( model ) rescue raise
end
binary = Binary.new.upload( model.upload )
if model.respond_to?( ‘after_upload’ )
model.after_upload( model, binary ) rescue raise
end
binary = binary.store()
model.binary_id = binary.id
model.active = 1
rescue => e
#
# Because we’re raising an exception, Rails will rollback
# the binary save operation at the database level.
#
File.delete( File.join( Rails.root, binary.path ) ) if binary
#
# Rethrow any exception that was raised.
#
raise e
end
end
What I Like
- It should take very little effort to support a new file type.
- The view and controller for a subtype doesn’t deviate very far from what’s provided by scaffolding. That makes for easy reading for new developers.
- The controller really has very little to do. The work is done down the stack by the models. I love a skinny controller.
What I Don’t Like
- I’d prefer to add the upload virtual attribute to the Binary class so I don’t have to add it to each new subtype, but doing so made the views more complex. Overall, it just feels like less effort to do it this way.
- Although it will take very little to support a new file subtype, it will require some effort (updating the observer, including the attribute accessor, etc.). Ideally, I’d like to have it be a no-effort kind of endeavor. That’s probably just wishful thinking.
It should be fairly clear from the title and intro that I’m on the (very) short side of the learning curve with Ruby & Rails. It’s entirely possible that I’ve made a mockery of any number of best practices or conventions. Constructive criticism is welcome. I’ve learned a lot by doing this, input from experience is never a bad thing.
ruby on rails
