[GE users] [ANN] rq-2.3.3

ara.t.howard at noaa.gov ara.t.howard at noaa.gov
Fri Jun 9 04:06:30 BST 2006

NOTE: hopefully it's ok to post this here.  rq fills such a different niche
than sge i assume it is.   please contact me offlist if you have feelings

   rq v2.3.3

   rq (queue | export RQ_Q=q) mode [mode_args]* [options]*



   ruby queue (rq) is a zero-admin zero-configuration tool used to create
   instant unix clusters.  rq requires only a central nfs filesystem in order
   to manage a simple sqlite database as a distributed priority work queue.
   this simple design allows researchers to install and configure, in only a
   few minutes and without root privileges, a robust unix cluster capable of
   distributing processes to many nodes - bringing dozens of powerful cpus to
   their knees with a single blow.  clearly this software should be kept out of
   the hands of free radicals, seti enthusiasts, and one mr. j safran.

   the central concept of rq is that n nodes work in isolation to pull jobs
   from an centrally mounted nfs priority work queue in a synchronized fashion.
   the nodes have absolutely no knowledge of each other and all communication
   is done via the queue meaning that, so long as the queue is available via
   nfs and a single node is running jobs from it, the system will continue to
   process jobs.  there is no centralized process whatsoever - all nodes work
   to take jobs from the queue and run them as fast as possible.  this creates
   a system which load balances automatically and is robust in face of node

   although the rq system is simple in it's design it features powerful
   functionality such as priority management, predicate and sql query , compact
   streaming command-line processing, programmable api, hot-backup, and
   input/capture of the stdin/stdout/stderr io streams of remote jobs.  to date
   rq has had no reported runtime failures and is in operation at dozens of
   research centers around the world.


     - fixed bug in updater related to io files (see tmp_stdin in jobqueue.rb)
     - [IMPORTANT] removed feature where stdin jobs could contain comments ('#').
       input lines are now taken literally __except__ blank lines, which are
       still ignored.
     - added ability to dump stdin/stdout/stderr for any job

         rq q stdout 42

         cat jids | rq q stdout -

     - added ability to dump stdin/stdout/stderr pathnames for any job
         rq q stdout4 42

         cat jids | rq q stderr4 -
     - rotate was made more robust and tolerates nfs errors to some degree


suffering increases your inner strength.  also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list