Caching user records when using Authlogic

Today, I ran across another form of this issue with Authlogic. In short, every request in a Rails application using Authlogic with a User model that includes a last_request_at column will cause the updated_at column to change. This can break caching that is based on the last time a user was updated.

last_request_at is one of Authlogic’s magic columns and is useful if you want to track users that might be logged in. However, by including this column, every controller request will touch the user record and change updated_at.

If you add last_request_at to your User model, you’ll start seeing this in your application logs:

UPDATE “users” SET “last_request_at” = ‘2014-03-12 20:59:48.314863’, “updated_at” = ‘2014-03-12 20:59:48.320481’

However, if you’ve added e-tags to a controller based on the updated_at timestamp, you’ll now have broken your cache control.  For instance, this will no longer work:

To fix caching based on the User’s updated_at timestamp with Authlogic, you can add your own timestamp and maintain it whenever the User record changes. Here is what I came up with:

  1. Add a new column, user_updated_at, to the user model
  2. Whenever a relevant column changes on user, set user_updated_at to Time.now
  3. Change the cache control to be based on user_updated_at instead of updated_at
Here is what that looks like in the User model:


And in the controller:


This made my controller request caching work as expected.  How would you handle this?

Quick local database saves and restores with postgres

I recently added this rake task to our project. It performs fast database saves and restores using another postgres database. This is a very handy rake task to have around if you need to quickly go back to a previous version of your database and the normal create/restore process is too slow (for instances, restoring using a scrubbed database backup hosted on S3)

There is a lot going on here, but it is more or less wrapping this command, which creates a database from another

with some helpers to cut existing connections and automate the backup.

I’ve used this for quite some time, but with a flash drive, I can restore an 8.7 gig database in about 25 seconds instead of about 8 1/2 minutes from a local postgres backup.

To use it, it is simply:

Its a handy command to have in the toolbox. Whats in your toolbox?

Find the most popular capitalization of a word or phrase

Today, I needed to find the most popular capitalization of user entered product brands. Previously, I’d just used lowercase brands, something like this:

However, this was returning “whole foods” instead of “Whole Foods”, which was considered more desirable.

Using window functions and common table expressions, this is what I came up with:

Given this data:

Here is how it works:

The first part of the query gives us the total count for each brand, grouped by lower case, and for each unique capitalization of that brand provides a count and row number.

In this query, the rows with row_number == 1 are all the unique capitalizations

Of those, we want the ones with the highest count. Again, we partition with another common table expression. This time, ordering by the count per unique capitalization to ensure that the first row for each group was the most popular

From here, we just pick the first row for each brand and get the most popular capitalization

Simple enough eh?

Messy rails code demonstrating this and generating the tables in this post

More about window functions

More about common table expressions

Adding Process ID and timestamps to the Rails Logger

For the last few days, I’ve been trying to debug a race condition between several Delayed::Job workers.  After looking at log files for many hours, it became very frustrating to not know which one of my 3 workers was writing which statement.  I finally found this post which almost did what I wanted.

To add process ids and timestamps to your Rails logs, you can add this as a Rails initializer (config/initializers/log_formatting.rb for instance) and restart your application / workers:

config/initializers/log_formatting.rb:

Before:

Log formatting before adding process ids and timestamps

After:

Log format after adding process IDs and Timestamps


How to write a simple Rails gem

Today, I wrote my first Ruby on Rails gem.  It was a very simple refactoring of our code that I undertook when I needed to add the same functionality to a new model and I decided to do it as a gem instead of keeping it within our project.  I wanted to see how it worked and this is what I ended up with.  You can follow the Rails Guide, but it didn’t cover everything I wanted to do.

Create a repository for your gem.

You can do this on github, like I did for my gem, locally, or somewhere else.  

Create a skeleton gem

The rails guide for creating a plugin has this, but the details in the guide are thin.

Rails 3.1 ships with a rails plugin new command which creates a skeleton for developing any kind of Rails extension with the ability to run integration tests using a dummy Rails application.

I used this to create a skeleton for my plugin:

This creates a skeleton gem with a dummy rails application you can use for testing your gem.

Switch to rspec

I’ve used rspec for all my rails testing and I wanted to use it with my new gem as well.  However, the generator creates a test-unit dummy application out of the box.  This StackOverflow question had a good answer that I used to switch from test-unit to rspec.

  1. Add rspec as a development dependency in your gemspec
  2. Bundle Install
  3. Convert from test-unit to rspec
  4. Modify spec_helper.rb with code taken from test_helper.rb
  5. Run the tests
  6. Commit the skeleton gem to source control

Author your gem

At this point, you have a skeleton gem that you can use to write your code.  The gem I wrote added some behavior to ActiveRecord models, so I started out by generating some models in the dummy rails application located in spec/dummy and using test-driven development to build my gem.  

Try it with a real project

The tests you author along with your gem are very helpful, but a time will come when you want to try your gem on your local file system with a real project.  You can include a gem from the local file system with this in your gemfile:


Squash your commits

When you’ve got your gem working and you are ready to publish it, you may want to squash all your commits to the repository into a single commit.

Take a look at http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html or search the web on how to squash multiple commits to a single commit, or try it with git rebase -i 

Benefits of Standing Desks

One of the perks of working from home is that I have total control over my work environment: chair, desk, computer, climate, light, etc*.  However, like many professionals, I sit for a lot of the day; 45+ hours a week, 48+ weeks a year for the past 12 years.  This isn’t healthy or natural.  I’ve been pretty good about getting exercise regularly, but since 45, or more, of my 120 waking hours during a week are spent working, there is a great opportunity to lead a healthier lifestyle with simple changes during work; small, simple sustained improvements can have a huge overall impact on your lifestyle and goals.


Many non-cycling fans may not be aware of Great Britain’s total dominance in track cycling during the 2012 olympics.  They won 7 out of 10 gold metals along with an additional silver and bronze; an incredible performance.  One of the ways that Great Britain dominated the track was taking by taking total control over their cycling ecosystem and finding lots of little improvements that, when added together, make a big difference.

Dave Brailsford, the performance director for Great Britain cycling and general manager of the top tier cycling team, Team Sky, sums up this philosophy as “The Aggregation of Marginal Gains”.  Any one gain in particular isn’t significant, but when added together, they are.  Along those lines, earlier, I mentioned that I use a crontab + growl to purposefully and gently interrupt my day with pushups or sit-ups.  Another way that I’ve found to make a small improvement in my daily life is with a standing desk.

Here are some of the benefits I perceive of a standing desk:

  1. Standing instead of sitting burns about 38% more calories.  The 2011 Compendium of Physical Activities lists sitting at a desk working and typing (code 09040) as 1.3 METS and standing working (codes 07041, 09070, 09055) as 1.8 METS.  MET is the ratio of the work metabolic rate to the resting metabolic rate and 1.0 MET is roughly equivalent to the energy cost of sitting quietly.
  2. It changes up the work day.  For me, working from home, sitting in one place and not seeing anybody in person all day gets boring.  I go to coffee shops sometimes and move around the house a little bit, but a standing desk gives me another place to work.  My standing desk also looks out a window which encourages me to shift my focus off my screen occasionally which is good for my eyesight.
  3. I move around more.  I pace while I’m on the phone, I shift my weight back and forth and overall, I don’t have sore muscles at the end of the day from being in the same position.  I do have sore muscles sometimes, but because a standing desk is hard work, not from bad posture. 
A standing desk can be cheap and easy to hack together.  Mine is setup with an extra wire shelving unit.  This made it easy for me to get the height adjustments correct.  I have the bottom weighted down for stability and simply move my laptop from my sitting desk to the top of the unit whenever I want to use the standing desk.  A bluetooth keyboard and mouse go on the second shelf to make the ergonomics work.  
I don’t use my standing desk all the time, right now, I used it for 2 to 4 hours a day. Its a nice change of pace and burns a few extra calories, and who knows, maybe it will add a few years to my lifetime.  Maybe someday I’ll add a treadmill for a walking desk.
How do you stay healthy at work?  Have you tried a standing desk? What do you like about it?

References:

* One downside is that there is no dedicated cleaning staff.  Another is few noise barriers with three children downstairs.

Expanding multiple sections of a JQuery Accordion

I’ve loved using jQuery and jQuery UI on my current project.  They work, on all browsers, and they make things simple.  They are powerful and extensible – great tools. However, one thing that has really annoyed me is the jQuery UI accordion.  They have this snippet in their documentation:

NOTE: If you want multiple sections open at once, don’t use an accordion

This drives me nuts!  If you have to write a note about something that the control doesn’t do, bolded and in a larger font size in the documentation, because lots of people want to do that, or are asking for it, the control should probably perform that function.

This is exactly what I wanted to try out for one of our UI prototypes.  Usually, we wanted one section open at a time (the accordion behavior), but sometimes we want to see all 5 sections (say for printing or so you don’t have to remember the first section when looking at the last section.

I found this excellent example of how to hack the jQuery UI accordion to allow multiple sections to be expanded at the same time: http://jsbin.com/eqape.

I then added this bit of JavaScript to expand and collapse all the sections.  These are hooked up as click handlers for a couple of icons on the page.

Expand/Collapse All

function accordion_expand_all()
{
  var sections = $(‘.accordion’).find(“h3”);
  sections.each(function(index, section){
    if ($(section).hasClass(‘ui-state-default’)) {
      $(section).click();
    }
  });
}

function accordion_collapse_all()
{
  var sections = $(‘.accordion’).find(“h3”);
  sections.each(function(index, section){
    if ($(section).hasClass(‘ui-state-active’)) {
      $(section).click();
    }
  });
}
This improved the jQuery accordion significantly and I hope we see all this behavior native to the control sometime in the future.

How I backup my photos

As my wife will quickly let you know, we lost some important photos of a trip we took due to a hard drive crash.  Since then, I’ve become very paranoid about how I store and back up our photos and videos.  Here is how I now back my photos and videos.

I’ve got all our photos and videos on a desktop machine with three hard drives.  The primary hard drive is a smallish SSD that has the operating system and not much else.  The other two drives are 1.5 TB spinning disks in a software-based raid 1 (mirroring) configuration.  At the end of each month I:

  • Download all my photos and videos to the desktop
  • Use robocopy to backup the spinning disks and important files from the SSD to a 1.5TB external HD
  • Swap the external HD I just backed up with an offsite external HD that has last month’s backup
This gives me good confidence that I’ve got my important documents backed up reliably.  At anytime, a document is on at least 3 hard drives (one of which is in a different physical location) and I’ve got last month’s backup in case I delete something or need an older version.  
There are still some problems with this scheme:
  1. Its not automatic
  2. I have to realize I deleted something before I make the next backup
However, its relatively cheap (you need 4x the hard-drives you might otherwise need) and it works. I’ve had to upgrade from 300 GB disks to 1.5 TB and had no problems restoring my backups.
How do you backup your important documents?  What am I missing that I should be doing?

Validating a User’s Age in Rails

We ran across an interesting bug in our application recently: a user who was 18 years old exactly could not sign up for our service.  This happened only in our production environment (not staging, development or test) and when we manually tested a number of dates, a user who was 18 years and 7 days old could sign up, but a user who was 18 years and 6 days old could not.

Here was our model validation:

user.rb

validates_inclusion_of :birthdate, :in=>Date.new(1900)..Time.now.years_ago(18).to_date, :message=>’You must be 18 years or older’

The problem? That is cached when the server starts.  It turns out, we had last deployed to production 7 days ago.  We didn’t see the problem in any other environments because they get restarted so frequently.

The solution is simple: use a proc that evaluates the current date each time:

user.rb

validate :at_least_18

def at_least_18

    if self.birthdate
      errors.add(:birthdate, ‘You must be 18 years or older.’) if self.birthdate > 18.years.ago.to_date
    end
end

I’m still not sure how to test this, but it is working.

How would you test this?  Where else do you have to use a proc in Rails that won’t show up locally?

Improve yourself with crontab and growlnotify

Its been a very long time since I last posted.  I’ve moved, changed jobs, major technologies (Windows / ASP.NET / C++ to Mac / Rails), had twin boys and am now working from home.  Working from home provides some great opportunities (family time instead of commute time being the big one).  I’ve found that I need help making sure that I take a break and step away from the computer occasionally during the day.  Also, I’m a skinny ex-cyclist who doesn’t like to go to the gym, but does want to be stronger.  Enter GrowlNotify and crontab to the rescue.


I use Growl and GrowlNotify for all sorts of things in my development work.  Anytime I kick off a long process (tests, database restore, etc), I use growlnotify to let me know that task is done.  That way, I don’t have to watch the process and can pick up immediately instead of possibly being distracted.

To provide me with a break every 30 minutes I’ve setup a cron job that sends me a message with growl.  Its very simple*:

~/.crontab

0,30 9-16 * * 1-5 /usr/local/bin/growlnotify -s -m “Time for some pushups or situps?” –image “/Users/johnnaegle/Pictures/growl_notify_pushup.jpg”

To let me easily change the timing, message or disable the notification easily, I put this in ~/.crontab (and some others) and added this to my .profile.  Whenever I launch a new terminal, the crontab gets updated with my latest list of timed notifications.
~/.profile

crontab < ~/.crontab

The result? I get a forced break (good for the eyes) and get stronger.  I’m already noticing a difference.
* every 30 minutes between 9am and 4pm on Monday through Friday, send me a sticky growl message with growl_notify_pushups.jpg as the icon.