HOWTO: Deal with Amazon’s “requested Availability Zone is no longer supported” error

In shutting down the AdGrok servers (talk about bittersweet…), I stopped the instances, but then remembered I wanted to shred the files first, so I clicked “start,” and was greeted by the following error:

The requested Availability Zone is no longer supported. Please retry your request by not specifying an Availability Zone or choosing us-west-1b, us-west-1c.

Assuming your instance was backed by an EBS volume (and there wouldn’t be any valuable state for the instance otherwise, so that should be a reasonable assumption), you’ll need to migrate your EBS volume to a different availability zone, and start a new instance there.

To move an EBS volume from one availability zone to another, you need to:

  1. create a snapshot of the EBS volume
  2. use the snapshot to create a new EBS volume in the destination zone
  3. attach the new EBS volume to an instance in the destination zone

You can do all these tasks through the AWS Management Console, in the “ELASTIC BLOCK STORE” sections.

It really is rocket science!

This last Friday I taught my “it really is rocket science” class to another hundred children (Kindergarten through 5th grade). Last year PTO Today wrote an article about Arts and Science day, and interviewed me. This year, the kids were great, I had tons of help, and the new rocket launcher designs let the kids experiment with different pressures and different launch angles. Good stuff!

HOWTO: Fix “rake/rdoctask is deprecated. use rdoc/task instead”

Seeing this?

rake/rdoctask is deprecated. Use rdoc/task instead (in RDoc 2.4.2+)

Edit your Rakefile and change these lines:

require 'rake/rdoctask'
Rake::RDocTask.new(:rdoc) do |rdoc|
 ...

to look like this:

require 'rdoc/task'
RDoc::Task.new do |rdoc|
  ...

You may need to add gem 'rdoc' to your Gemfile, too. While you’re at it, you might want to add rdoc/ to your .gitignore, too

HOWTO: Wrangle the pg (postgresql) gem with macports and rvm

Seeing this?

Installing pg (0.11.0) with native extensions /Users/mrm/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/rubygems/installer.rb:533:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

        /Users/mrm/.rvm/rubies/ruby-1.9.2-p180/bin/ruby extconf.rb
checking for pg_config... no
No pg_config... trying anyway. If building fails, please try again with
 --with-pg-config=/path/to/pg_config
checking for libpq-fe.h... no
Can't find the 'libpq-fe.h header
*** extconf.rb failed ***

Wondering how or where to get this mythical pg_config? It’s part of the PostgreSQL package. If you’re running MacPorts, it’s easy:

sudo port install postgresql90

Then, following the instructions on the RVM website, run

gem install pg -- --with-pg-config=/opt/local/lib/postgresql90/bin/pg_config

Creating acts_as_… gems for Rails 3.1.x

There are a lot of posts on how to build rubygems. With Rails 3.1, though, they’re all old and busted.

The new hotness is built right into rails now. Just incant

rails plugin new APP_PATH

You’ll get:

  • a .gemspec and Gemfile to get started
  • a dummy rails app to integration test your new gem against
  • a (deprecated) RDoc rake task

When you run rake, you’ll see “rake/rdoctask is deprecated. Use rdoc/task instead (in RDoc 2.4.2+)”. To remedy, follow these steps.

The Edge edition of the Ruby on Rails Guides has more information.

Hierarchical Tagging with Rails 3 and Closure Trees

In rebuilding PhotoStructure on Rails, I was surprised that one of the most popular gem for trees used a nested set model. Nested sets are performant for reads, but for adding and deleting nodes, it’s extremely expensive — on average, half of the rows for your model acting as a nested set have to be updated for every add or delete. That’s crazy-talk! It requires a table-level lock for something that, at least for PhotoStructure, is a very common task. To top it off, the gem doesn’t even do the lock!

I then found the ancestory gem, which works by materializing the ancestral path as a string and storing that as a column. Bill Karwin’s excellent Models for hierarchical data presentation describes this as the Path Enumeration algorithm, which doesn’t have referential integrity, and relies on performant LIKE selects.

As the tag hierarchies in PhotoStructure are never moved, closure trees should prove to be ideal. It’s an excellent excuse to learn how to build and publish a rails plugin gem, so I’ve built a new gem to support closure trees: github.com/mceachen/closure_tree.

Update, May 24

1.0.0.beta1 released. All public methods have test coverage.

Update, May 25

1.0.0.beta2 and 1.0.0.beta3 were released, which added find_or_create class and instance methods, ancestry_path, and root instance methods. Documentation is on the README and in the rdocs.

Update, May 29

1.0.0.beta5 is released, which cleaned up the ancestor and descendant relationships to use has_and_belongs_to_many, and adds leaves class and instance methods.

Update, Oct 26

2.0.0.beta1 is released, which added the :dependant option, switched from an embedded dummy rails app for testing to rspec, and much better test coverage (including tests and fixes for a couple reported issues) under sqlite, MySQL, and PostgreSQL.

Update, Nov 27

3.0.0 is released, which supports polymorphic trees.

Sandbox telegrams, or, how your Chrome extension can interact with page content scripts

In AdGrok’s GrokBar, we inject a “heads-up display” on pages that the user is advertising. The heads-up display is actually an iframe that’s positioned within a browser-extension-injected div, and that iframe renders content from our secure server farm. You can see a demo video here (and see that we truly spared no expense on the voice-over talent!).

I wanted to make our extension’s button-click incant a javascript method that was inside the GrokBar’s iframe. I found this horrible hack, but every time a javascript timer scrapes a hidden DOM element, or mucks with the URL fragment of an iframe src in order to send messages, the code gods kick a puppy.

Continue reading

Run a RAID6 on Amazon EBS For Fun and Profit

With all this badmouthing Amazon’s Elastic Block Store, I wanted to share how we’ve set up our MySQL server for AdGrok.

Step 1: Create a bajillion tiny EBS volumes

We’ve got a bunch of performance data in MySQL, so our database is in the tens-of-gigabytes size. I didn’t want to worry about rebuilding the RAID, so I used the AWS console to build 8 15GiB volumes.

Why eight volumes? Because I wanted to use RAID6.

Continue reading