Mongrel_cluster not starting after hard reboot

Does the following error sound familiar?

** !!! PID file log/mongrel.pid already exists.  Mongrel could be running already.  Check your log/mongrel.log for errors.
** !!! Exiting with error.  You must stop mongrel and clear the .pid before I'll attempt a start.

It usually happens when the server crashes. After that you need to ssh into it, remove the mongrel pid files and start the cluster manually. No more.

I assume you have mongrel_cluster setup properly, ie: project’s config file is in /etc/mongrel_cluster and the mongrel_cluster script has been copied from:
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-*/resources/
to the /etc/init.d directory. You need to edit the /etc/init.d/mongrel_cluster file:

Change this two bits:

start)
  # Create pid directory
  mkdir -p $PID_DIR
  chown $USER:$USER $PID_DIR

  mongrel_cluster_ctl start -c $CONF_DIR
  RETVAL=$?
;;

and

restart)
  mongrel_cluster_ctl restart -c $CONF_DIR
  RETVAL=$?
;;

to

start)
  # Create pid directory
  mkdir -p $PID_DIR
  chown $USER:$USER $PID_DIR

  mongrel_cluster_ctl start --clean -c $CONF_DIR
  RETVAL=$?
;;

and

restart)
  mongrel_cluster_ctl restart --clean -c $CONF_DIR
  RETVAL=$?
;;

respectively.

Adding the --clean option makes the mongrel_cluster_ctl script first check whether mongrel_rails processes are running and if not, checks for existing pid files and deletes them before proceeding.

You must be using the mongrel_cluster version 1.0.5+ for it to work as advertised (previous versions were buggy). To upgrade do:

gem install mongrel_cluster
gem cleanup mongrel_cluster

Here’s the related mongrel_cluster changeset.