Inheriting Attributes in a Tree Data Structure

I just published a new gem, inherited-attributes, for Active Record that works with the Ancestry gem to provide a tree data structure the ability to inherit attributes from their parent node, or farther up the tree.  We’ve been using this technique for a long time to support configuring our multi-tenant application.

Once the gem is installed, its very simple to configure:

ActiveRecord::Schema.define do
  create_table :nodes, :force => true do |t|
    t.string :name
    t.string :value
    t.string :ancestry, :index => true
  end
end

class Node < ActiveRecord::Base
  has_ancestry

  inherited_attribute :value
end

From there, you can access the effective attributes which look up the tree ancestry to find a value to inherit.

root       = Node.create!
child      = Node.create!(:parent => root, :value => 12)
grandchild = Node.create!(:parent => child)

root.effective_value       # nil
child.effective_value      # 12
grandchild.effective_value # 12 -- inherited from child

There are more options and examples in the gem, including has-one relationships, default values and support for enumerations.

We’ve found it helpful and writing a gem made this code much easier to test.  What code do you have that would be easier to test as a gem or would be useful to others?

List EC2 Instances in Hubot

I like slack.  I like automating things. I hate email.  Therefore, I like Hubot.  I’ve written Hubot scripts that integrate with our ticketing system and launch Skype calls to everybody in the channel.  Earlier this week, I added autoscaling notifications to Slack when our servers automatically scale up or down.  To go along with this, I wanted to see all the servers we’re currently running in EC2.

I found some scripts that did roughly what I wanted, but those scripts 1) Did more than I wanted and 2) didn’t have the formatting I wanted for the ls command and 3) didn’t filter in the way I wanted.  Based on yoheimuta’s scripts I created a new npm package, hubot-ec2, that just lists instances.

Installation is straight forward in your hubot instance:

npm install --save hubot-ec2
npm install

After installing, set 3 environment variables

  HUBOT_AWS_ACCESS_KEY_ID="ACCESS_KEY"
  HUBOT_AWS_SECRET_ACCESS_KEY="SECRET_ACCESS_KEY"
  HUBOT_AWS_REGION="us-east-1"

The keys used should be a user with the IAM Policy AmazonEC2ReadOnlyAccess.

Once thats all set, deploy your bot and you’ve got a new command:

hubot ec2 ls

You can filter by instance name:

hubot ec2 ls i-abcd1234

or by tag name:

hubot ec2 ls production-*

It will look something like this when its all wired up:

 

Do you use hubot and slack?  What have you done to make it awesome?

Posting Amazon Autoscaling Notifications to Slack

Dev-ops has become a little bit of a buzz-word, but its something I’ve come to embrace and enjoy; automating our infrastructure and deployments has streamlined our software development lifecycle.   One piece of our infrastructure is AWS Autoscaling.  With AWS Autoscaling, we can easily add new servers in response to shifting usage patterns.  For instance, during the night, we might drop down to 1 server as our site quiets down and as usage grows during the day, automatically scale up to as many as 6 servers.  Its been a useful cost savings measure and its also forced us to get our server provisioning correct through Ansible.

I wanted to know, via slack, when we scaled up our down.  AWS doesn’t provide this functionality directly, but with a few hoops, it wasn’t too hard to add.

  1. Setup Amazon Simple Notification Service (SNS) to receive messages
  2. Use Amazon Lambda to transform SNS messages to Slack webhooks
  3. Configure Autoscaling to send SNS messages
Because we need the Lambda function before we setup the SNS endpoint, we’ll do that first

Setup Lambda Function

AWS Lambda is a snippet of your code, running somewhere in the cloud.  You don’t manage the server and you only pay for the time you use.  Lambda currently supports Java, Python and Node.  We’ll write a simple node script that sends an SNS message to a Slack incoming webhook.
  1. Login to your AWS console and find the Lambda functions
  2. Create a new Lambda
  3. Skip the blueprint selection, start with a blank script
  4. Give your lambda function a name (autoscaling2Slack), optional description and pick the node runtime (4.3 as of this writing)
  5. Configure the script below as the lambda function with your slack webhook URL
  6. Create a new IAM role for the lambda function (basic execution role)

To test your lambda you can use this SNS message.  To view the real SNS messages, you can add some console.log statements to the lambda function and view them when autoscaling runs.

If you’ve set it up correctly, you should see the test message come through to slack:

 

Setup SNS Endpoint

With the Lambda in place, the SNS endpoint can be created.
  1. Login to your Amazon Console and find the SNS and create a new topic (slackSNS).
  2. Create a subscription to send to your Lambda function.
When you’ve got it setup, it should look something like this:

Send SNS messages when autoscaling occurs:

  1. Login to your EC2 console.
  2. In your autoscaling groups, add a new notification to your SNS topic.
When its setup, the ASG notifications should look something like this:

Wrapping up

I like slack and I hate email.  This lets me see autoscaling notifications from an SNS message that doesn’t go to email.  Hopefully you’ve found this useful.

References

 
I started with this, which sends Elastic Beanstalk notifications to slack: https://medium.com/cohealo-engineering/how-set-up-a-slack-channel-to-be-an-aws-sns-subscriber-63b4d57ad3ea.  The process for AWS autoscaling is _almost_ the same, but the SNS payload is different.

Popups on your Popups

We’ve been using Bootstrap 3 for a lot of the newer screens in our application – particularly our internal, administrator facing screens.  It has been a great framework.  However, the Bootstrap Modals do leave a fair amount to be desired.  You have to write a lot of boilerplate code and then bootstrap has this in their documentation:

Be sure not to open a modal while another is still visible. Showing more than one modal at a time requires custom code.

To avoid writing boilerplate, we’ve used the monkey-friendly Bootstrap Dialog javascript library from nakupanda.  This makes showing a Bootstrap Modal as simple as:

 

Okay, but what if you want to show a modal on top of another modal?  That is custom code!  Oh look, custom code:

 

A few things are going on here:

  1. When the modal starts to show, set the visibility to hidden.  Bootstrap animates the modal coming in from the top of the screen.  However, the second modal animates in behind the first.  So hide the modal until its completely shown.
  2. To make the second modal come to the foreground, the z-indexes need to be adjusted.  If you dig into Bootstrap 3, you’ll find the z-index of the dialog is 50000 and the background is 1030.  So the new modal and back drop need to be in front of those.
  3. The timeout of 100 ms is probably not necessary, but in my application I am displaying an overlay spinner when the link is clicked.  The 100ms makes the overlay show for a minimum amount of time.

Modals on top of modals are a kludge, and they aren’t a great design choice, but when you need them, you need them.

Barf: I know we need the money, but…
Lone Starr: Listen! We’re not just doing this for money!
Barf: [Barf looks at him, raises his ears]
Lone Starr: We’re doing it for a SHIT LOAD of money!

Maybe in the future, Bootstrap will make multiple modals easier, but I think its justifiable that multiple modals requires custom code.

Capistrano Deploys without Swap

I work on a Ruby on Rails application that is deployed with Capistrano 3 to Amazon Web Services.  We monitor our site performance with New Relic.  About a month ago, we noticed that our deploys were causing a delay in request processing and a drop in Apdex.

Here is an example of what we saw during a deploy.  The blue vertical lines are the deploy times and the green bar is how long a request spent waiting to be processed.

When we dug into it, we found that our servers were going into memory swap during the deploys.  When we deploy with Capistrano, a new Rails process is started to pre-compile the assets.  This pushes the memory usage over the physical memory limits.   Here is the key graphic from New Relic.  Note the swap usage just after 3:00 pm and the disk I/O at the same time.

You can see in the graph above how memory usage drops after the deploy so the solution to this was pretty straight forward: restart the servers first.

We’re using Puma as our web server, so we added these lines to our deploy file.  This causes a phased-restart to be sent before the deploy, freeing memory and allowing the asset compilation to have enough memory to run without using swap.  Since Capistrano is based on Rake, its important to re-enable the phased-restart task after its run, otherwise it will only be run once.

Now our deploys run without causing requests to be queued.  What tricks do you have for zero-impact deployments?

Labeling Rails Enums

Rails 4.1 introduced ActiveRecord Enums, a handy little feature that lets you store an integer in a database column, but use symbols or strings in your code.  Until today, I’ve been presenting these in select boxes with code like this:

This generates a perfectly acceptable select box if you don’t need internationalization and the enum names you’ve picked are good enough to present to the user.  If thats not the case, or you to change the presentation without changing the code, a new helper method leveraging Ruby’s internationalization (i18n) features can be a good approach.

Keep reading for how I changed the presentation of enumerations in a drop down without changing my models or renaming the enumeration names.


Here is how we can provide alternate labels for some of our drop down values.

First, create a new helper that will pass each enumeration name through i18n to perform a label conversation

Second, change your form to call the helper

Third, provide the translations.

Before translation:

After translation:

A few notes:
  • If the translation is missing, the code’s default is to fall back to the titleized text.  If you want to omit options where the translation is missing,  You can modify the methods to call enum_to_translated_option with a default of blank and the blank options will be removed from the select.
  • enum_to_translated_option is useful in a show template.  
  • It may be useful to sort the options after translation.  That is an easy addition to the helper.

Is this useful for you?  Is there a better way?  Let me know!

A fun IE9 surprise

Every time I have to debug an IE9 (or lower) issue, a puppy dies. You don’t want puppies to die do you? Browse happy: http://t.co/v4MCw2MOkH
— John Naegle (@johnnaegle) November 11, 2014

As much as I would like to avoid having to deal with IE8 and IE9, sometimes that is just a requirement of the job. Today, I ran across a fun JavaScript bug.

Can you spot it below?

In IE9, item is a function and since the loop left out the var keyword before item, there wasn’t a local variable named item in the block.  The fix is simple, finding the bug, not so much.

Do yourself a favor and save a puppy, get a modern browser, and Browse Happy.

Removing milliseconds in JSON under ActiveRecord 4.0

Rails 4.0 introduced a small bug in JSON generation with this pull request. The output format for times (ActiveRecord::TimeWithZone) in JSON changed to include milliseconds. Sounds good right? Well, not if your API clients crash trying to parse milliseconds. Unfortunately, Rails 4.0 didn’t provide a configuration option for the timestamp precision in JSON output. What is a programmer under the gun to do? Upgrade to Rails 4.1 or get out your monkey and your patch and get to work?


First, lets see the bug in action. This test passes in Rails 3.2.15, but fails in Rails 4.0.4:

Simple enough. This is easier to see if you’ve got a Rails app:

Rails 4.1 adds a configuration option to default the time precision. In that version, you can set ActiveSupport::JSON::Encoding.time_precision = 0. However, if you are on Rails 4.0 for a bit, you can monkey patch AtiveSupport::TimeWithZone to go back to the Rails 3.2 implementation:

Normally, it seems the Rails team does a better job than this. Fortunately, with Ruby we’ve got the openness to go back to the old behavior in a pinch.

Model is a poor scope name in Ruby on Rails 4

Upgrading from Rails 3.2 to Rails 4.0 is not a trivial task. Sure, there is a guide, but when you upgrade you’ll probably be upgrading a lot of your gems, maybe your jQuery and jQuery-UI versions and then there are all the undocumented unintentional changes that can cause you grief. If you’ve got a good test suite (you have one right?) you’ll catch a lot of these, but some will leave you scratching your head. This one tripped me up for a while: we had a very simple scope stop working in Rails 4.

The scope model below works in Rails 3.2, but fails in Rails 4 when used with an association:


Here is the failure:

In the second query, post.comments.model.count the query has lost the comments scope as well as the conditional defined in our model scope and it counts all comments in the system.

The fix is quite easy, just change the name of the scope to something like not_index.

This seems to be change in associations that is most clearly illustrated by this:

I could never quite figure out where in activerecord this changed, but model was not a great choice for a scope name. This executable gist has everything in this post if you want to explore this more.

What surprises have you found with Rails 4?

Rails: updating an association through nested attributes does not touch the owner

I was a little surprised to discover that when you update an association through nested attributes, it won’t touch the parent record.  It makes sense when you consider that Rails is optimizing by not writing records that have not changed, but if were using updated_at on the model for caching you may be surprised.

For instance:

With these models:

This code will not change updated at on the post record:

As shown in the log:

The fix is very simple. Just touch the post record from the comment if you need updated_at to change on posts.

And now our post is updated:

Of course there are other ways to do this…

The code for this is on github.