Lazy Mixin

# Before you ask--yes, this has a very specific use case and is not
# generally applicable to the problem of fine-grained lazy evaluation.
# But it does *exactly* what I need it to do. =)

# Allows attributes to be declared as lazy, meaning that they won't be
# computed until they are asked for. Just mix this module in:
#
#   class Foo
#     include Lazy
#     ...
#   end
#
# To specify a lazy reader:
#
#   lazy_reader :att
#
# Then, define a method called __bake__ that computes all your lazy
# attributes:
#
#   def __bake__
#     @att = ...
#   end
#
# That's it! (Tom Preston-Werner: rubyisawesome.com)
module Lazy
  module ClassMethods
    def lazy_reader(*args)
      args.each do |arg|
        define_method(arg) do
          self.__prebake__
          instance_variable_get(('@' + arg.to_s).intern)
        end
      end
    end
  end
  
  def __prebake__
    return if @__baked__
    self.__bake__
    @__baked__ = true
  end
  
  def self.included(base)
    base.extend(ClassMethods)
  end 
end

# A pedagogical class with a lazy attribute
class Foo
  include Lazy
  
  attr_reader :id
  lazy_reader :body
  
  def initialize(id)
    @id = id
  end
  
  def __bake__
    puts "__baking__"
    @body = "The 30th fibonacci is #{self.fib(30)}"
  end
  
  def fib(n)
    return 1 if n == 0 || n == 1
    return fib(n-1) + fib(n-2)
  end
end

t = Foo.new('abc')

puts t.id      # => (immediate) "abc"
puts t.body    # => (short pause) The 30th fibonacci is 1346269
puts t.body    # => (immediate) The 30th fibonacci is 1346269

1 comment

Hash Defaults and Path of Madness

This is a little story about innocent Ruby mistakes and how they can ruin your day. One of our interns here are Powerset had some code that used default hash values.

12	h = {}h.default = []

Fair enough. Then the code did this:

h[:foo] << 'foo' # => ["foo"]

Which happily returns in exactly the expected fashion. We can even check the value to make sure:

h[:foo] # => ["foo"]

But then later on we look more closely at the hash and WTF it's empty!

h # => {}

Madness! Until you realize that a hash's default value is only used as the return value for a hash lookup of a non-existent key.

12	h.default # => ["foo"]h[nil] # => ["foo"]

And then you can only laugh once you look back at the code that seemed to work and see exactly why it was so deceiving.

3 comments

god 0.4.0 released

Progress on god is moving along as quick as ever. This release adds a bunch of new features and bug fixes. Most interestingly you'll find several useful new command line functions:

god status prints out the status of each Watch
god log shows realtime logs for a specific Watch (even if you don't have god logging to file)
god load loads or reloads a config file into a running god instance
god terminate stops all Watches and then stops god (useful when testing your setup)

The logging system has been beefed up with proper timestamps and criticality levels. Log messages are more complete overall. You can also get the STDOUT/STDERR of a god-daemonized process written to a log file by specify 'w.log = ' in your Watch config.

If you let god daemonize your process for you, there's no need to provide a stop command. A default killing lambda will take care of gracefully (or not so gracefully if necessary) stopping your god-daemonized process.

The validity of your config file is checked better than previous versions to point you to the problem area of your config.

The bug that prevented group control from working has been fixed so you can now start/stop/etc groups of Watches.

Updated documentation is now available on the website:

http://god.rubyforge.org/

WHAT IS GOD?

God is an easy to configure, easy to extend monitoring framework written in Ruby.

Keeping your server processes and tasks running should be a simple part of your deployment process. God aims to be the simplest, most powerful monitoring application available.

DISCLAIMER

God is still very young, I'd love to get feedback and bug reports, but I do not yet recommend you use it for mission critical tasks. I personally use it in production but then I'm a daring fellow.

INSTALL

sudo gem install god

FEATURES

Config file is written in Ruby
Easily write your own custom conditions in Ruby
Supports both poll and event based conditions
Different poll conditions can have different intervals
Easily control non-daemonized processes

EXAMPLE

The easiest way to understand how god will make your life better is by looking at a sample config file. The following configuration file is what I use at gravatar.com to keep the mongrels running:

# file:      gravatar.god
# run with:  god -c /path/to/gravatar.god
# 
# This is the actual config file used to keep the mongrels of
# gravatar.com running.

RAILS_ROOT = "/var/www/gravatar2/current"

%w{8200 8201 8202}.each do |port|
  God.watch do |w|
    w.name = "gravatar2-mongrel-#{port}"
    w.interval = 30.seconds # default
    w.start = "mongrel_rails cluster::start --only #{port} \
      -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
    w.stop = "mongrel_rails cluster::stop --only #{port} \
      -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
    w.grace = 10.seconds
    w.pid_file = File.join(RAILS_ROOT, "log/mongrel.#{port}.pid")
    
    w.behavior(:clean_pid_file)

    w.start_if do |start|
      start.condition(:process_running) do |c|
        c.interval = 5.seconds
        c.running = false
      end
    end
    
    w.restart_if do |restart|
      restart.condition(:memory_usage) do |c|
        c.above = 150.megabytes
        c.times = [3, 5] # 3 out of 5 intervals
      end
    
      restart.condition(:cpu_usage) do |c|
        c.above = 50.percent
        c.times = 5
      end
    end
  end
end

DOCS

Detailed documentation is available at http://god.rubyforge.org/

CHANGES

== 0.4.0 / 2007-09-13

Major Enhancements
- Add the ability for conditions to override transition state (for exceptional cases)
- Implement dynamic load of config files while god is running (god load )
- Add ability to save auto-daemonized process output to a log file
- Add robust default stop lambda command for auto-daemonized processes (inspired by _eric)
- Add status command for god binary (shows status of each watch)
- Create proper logger with timestamps
- Add log command to god binary to get real time logs for a specific watch from a running god instance
- Add terminate command for god binary (stop god and all watches)
Minor Enhancements
- Enforce validity of Watches
- Enforce that God.init is not called after a Watch
- Move pidfiledirectory creation and validation to God.start
- Remove check for at least one Watch during startup (now that dynamic loading exists)
New Conditions
- Tries < PollCondition - triggers after the specified number of tries
- Add :notifywhenflapping behavior to check for oscillation [kevinclark]
- Add :degrading_lambda condition. [kevinclark] It uses a decaying interval (1/2 rate) for 3 cycles before failing.
Bug Fixes
- Use exit!(0) instead of exit! in god binary to exit with code 0 (instead of default -1)
- Command line group control fixed
- Fix cross-thread return problem

AUTHORS

Tom Preston-Werner Kevin Clark

4 comments

Powerset Interview

I was recently interviewed by my buddy Ryan Ferrier as part of a series of Powerset employee interviews for Steve Newcomb's blog. I know you think interviews are boring, so here's a little taste of mine to dispel any myths:

Rumor has it that you listen almost exclusively to techno music, is this true?

My dream is to code in a large, white, egg-shaped room wherein a large, omnidirectional speaker system is suspended at one locus, and an elaborate, 9 screen, multi-platform computing setup is suspended at the other locus. In this dream, I am sitting comfortably at the computer locus while thundering trance emanating from the opposite locus is parabolically focused upon my exact location. Then I could get some *real* coding done.

Rumor also has it that when listening to said techno music, you get in a trance-like state for hours on end and transform into a coding machine. Could you describe this altered state of being for us?

Remember the guys from "The Matrix" that could look at those screens of waterfalling green glyphs and see real world objects in them? It's nothing like that. That's impossible and ridiculous. If you've ever sat zazen and meditated on some ancient koans, that would be another good example of what it's not like. It's also not like watching your dog eat peanut butter. I'd keep going but it would take infinitely long to describe everything that it's not, so I'll have to stop and leave you mystified by the enigma of it all.

Continue reading the full Tom Preston-Werner Interview.

0 comments

god 0.3.0 released

Plenty of big changes and improvements in this release of god. Most noticeable is a simplification of the config file. Sadly, we must say goodbye to the much loved God.meddle and promote God.watch to top level. This change allows you to easily load in other god config files and have them work as expected. There's even a God.load that takes a glob-style file path string and expands it to load multiple files.

PID file support is now baked in so you don't have to set the pid_file attribute on conditions anymore. This also allows god to support processes that aren't already daemons. If you don't specify a PID file for the watch, god will auto-daemonize and keep track of your process for you! To use this feature, you must either run as root (pid files will be stored by default in /var/run/god) or set the pid_file_directory attribute in a God.init block (see docs).

For anyone that had problems installing the 0.2.0 release because of the events system not compiling, god 0.3.0 will now allow you to install without event support. Most systems are capable of supporting events, but it requires some dedication to get the headers in place and make sure your kernel is new enough. To determine if your god installation has event support you can run "sudo god -V" (this MUST be run as root on Linux, as netlink connector requires root permissions).

Using the god binary you can now control your watches from the command line. You can start/stop/restart/monitor/unmonitor watches at your demand.

New watch attributes now available: pid_file: sets the watch process' PID file location uid/gid: start processes as someone else (requires root access) group: assign a watch to a group (to control them all at once) autostart: prevent auto start on god start if false

Updated documentation is now available on the website:

http://god.rubyforge.org/

WHAT IS GOD?

God is an easy to configure, easy to extend monitoring framework written in Ruby.

Keeping your server processes and tasks running should be a simple part of your deployment process. God aims to be the simplest, most powerful monitoring application available.

DISCLAIMER

God is still very young, I'd love to get feedback and bug reports, but I do not yet recommend you use it for mission critical tasks. I personally use it in production but then I'm a daring fellow.

INSTALL

sudo gem install god

FEATURES

Config file is written in Ruby
Easily write your own custom conditions in Ruby
Supports both poll and event based conditions
Different poll conditions can have different intervals
Easily control non-daemonized processes

EXAMPLE

The easiest way to understand how god will make your life better is by looking at a sample config file. The following configuration file is what I use at gravatar.com to keep the mongrels running:

# file:      gravatar.god
# run with:  god -c /path/to/gravatar.god
# 
# This is the actual config file used to keep the mongrels of
# gravatar.com running.

RAILS_ROOT = "/var/www/gravatar2/current"

%w{8200 8201 8202}.each do |port|
  God.watch do |w|
    w.name = "gravatar2-mongrel-#{port}"
    w.interval = 30.seconds # default
    w.start = "mongrel_rails cluster::start --only #{port} \
      -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
    w.stop = "mongrel_rails cluster::stop --only #{port} \
      -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
    w.grace = 10.seconds
    w.pid_file = File.join(RAILS_ROOT, "log/mongrel.#{port}.pid")
    
    w.behavior(:clean_pid_file)

    w.start_if do |start|
      start.condition(:process_running) do |c|
        c.interval = 5.seconds
        c.running = false
      end
    end
    
    w.restart_if do |restart|
      restart.condition(:memory_usage) do |c|
        c.above = 150.megabytes
        c.times = [3, 5] # 3 out of 5 intervals
      end
    
      restart.condition(:cpu_usage) do |c|
        c.above = 50.percent
        c.times = 5
      end
    end
  end
end

DOCS

Detailed documentation is available at http://god.rubyforge.org/

CHANGES

== 0.3.0 / 2007-08-17

Fix netlink header problem on Ubuntu Edgy [Dan Sully]
Add uid/gid setting for processes [kevinclark]
Add autostart flag for watches so they don't necessarily startup with god [kevinclark]
Change command line call options for god binary to accommodate watch start/stop functionality
Add individual start/stop/restart grace periods for finer grained control
Change default DRb port to 17165 ('god'.to_i(32))
Implement command line control to start/restart/stop/monitor/unmonitor watches/groups by name
Watches can now belong to a group that can be controlled as a whole
Allow god to be installed (sans events) on systems that don't support events
Daemonize and handle PID files for non-daemonizing scripts [kevinclark]
Fix simple mode lifecycle gap
Remove necessity to specify pid_file for conditions
Change config file to use God.init and God.watch directly instead of God.meddle block
Move god binary command logic to main library
Enhance god binary with better reporting
Fix synchronization bug in Timer (reported by Srini Panguluri)
Add Lambda condition for easy custom conditions [Mike Mintz]
Add sugar for numerics (seconds, minutes, kilobytes, megabytes, percent, etc)
Add optional PID and log file generation to god binary for daemon mode
Add God.load to do glob enabled loading
Add -V option to god binary for detailed version/build info

AUTHORS

Tom Preston-Werner
Kevin Clark

9 comments

god 0.2.0 released

A lot of work has gone into god since the last release. Say goodbye to polling for process status--support for event based conditions via kqueue/netlink has been added (thanks Kevin Clark)! Only a few events are currently supported (most prominently process exit), but the way is now open for the addition of other events. A new, advanced syntax is available (in addition to the familiar simple syntax) that gives you full power over the new state based lifecycle of your Watches. Updated documentation is available on the website:

http://god.rubyforge.org/

WHAT IS GOD?

God is an easy to configure, easy to extend monitoring framework written in Ruby.

Keeping your server processes and tasks running should be a simple part of your deployment process. God aims to be the simplest, most powerful monitoring application available.

DISCLAIMER

God is still very young, I'd love to get feedback and bug reports, but I do not yet recommend you use it for mission critical tasks. I personally use it in production but then I'm a daring fellow.

INSTALL

sudo gem install god

note: currently tested only on Redhat Linux and Darwin (won't work on Windows)

FEATURES

Config file is written in Ruby
Easily write your own custom conditions in Ruby
Supports both poll and event based conditions
Different poll conditions can have different intervals

EXAMPLE

The easiest way to understand how god will make your life better is by looking at a sample config file. The following configuration file is what I use at gravatar.com to keep the mongrels running:

# file:      gravatar.god
# run with:  god start -c /path/to/gravatar.god
# 
# This is the actual config file used to keep the mongrels of
# gravatar.com running.

RAILS_ROOT = "/var/www/gravatar2/current"

God.meddle do |god|
  %w{8200 8201 8202}.each do |port|
    god.watch do |w|
      w.name = "gravatar2-mongrel-#{port}"
      w.interval = 30 # seconds default
      w.start = "mongrel_rails cluster::start --only #{port} \
        -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
      w.stop = "mongrel_rails cluster::stop --only #{port} \
        -C #{RAILS_ROOT}/config/mongrel_cluster.yml"
      w.grace = 10 # seconds
      
      pid_file = File.join(RAILS_ROOT, "log/mongrel.#{port}.pid")
      
      w.behavior(:clean_pid_file) do |b|
        b.pid_file = pid_file
      end

      w.start_if do |start|
        start.condition(:process_running) do |c|
          c.interval = 5 # seconds
          c.running = false
          c.pid_file = pid_file
        end
      end
      
      w.restart_if do |restart|
        restart.condition(:memory_usage) do |c|
          c.pid_file = pid_file
          c.above = (150 * 1024) # 150mb
          c.times = [3, 5] # 3 out of 5 intervals
        end
      
        restart.condition(:cpu_usage) do |c|
          c.pid_file = pid_file
          c.above = 50 # percent
          c.times = 5
        end
      end
    end
  end
end

DOCS

Detailed documentation is available at http://god.rubyforge.org/

CHANGES

== 0.2.0 / 2007-07-18

Rewrote innards to use a state and event based lifecycle
Basic support for events via kqueue (bsd/darwin) and netlink/pec (linux) [kevinclark]
Added advanced syntax (simple syntax calls advanced api underneath)
Condition returns have changed meaning. With simple syntax, a true return activates block
Updated http://god.rubyforge.org with updated simple config and new advanced config

AUTHORS

Tom Preston-Werner
Kevin Clark

21 comments

MySQL secrets: \G instead of ;

I'm a MySQL command line junkie. Even when I have a GUI frontend installed, I always find myself typing mysql into terminal when I need to peek inside the database. As such, one trick that has changed my life for the better is the \G statement terminator. Normally when you execute a query you get something like:

mysql> select * from queries order by id desc limit 1;
+-----+------------------------------------+--------------+------------+------------+
| id  | query                              | query_set_id | created_at | updated_at |
+-----+------------------------------------+--------------+------------+------------+
| 969 | when a stranger calls sound boards |          103 | 2007-07-09 | 2007-07-09 | 
+-----+------------------------------------+--------------+------------+------------+
1 row in set (0.00 sec)

That's ok, but for tables with more than a few columns you get a nasty bunch of badly wrapped output that's essentially impossible to decipher. If you replace the terminating semicolon with \G, you'll get this output instead:

mysql> select * from queries order by id desc limit 1 \G
*************************** 1. row ***************************
          id: 969
       query: when a stranger calls sound boards
query_set_id: 103
  created_at: 2007-07-09
  updated_at: 2007-07-09
1 row in set (0.00 sec)

Aaaaaaaaah! Now we're talking. You can even copy this output and paste it right into a YAML fixture in your rails app (with a little fixing of the indentation)!

4 comments