[GE users] SGE capability question
Dan.Templeton at Sun.COM
Fri Dec 14 16:28:21 GMT 2007
[ The following text is in the "ISO-8859-1" character set. ]
[ Your display is set for the "ISO-8859-10" character set. ]
[ Some special characters may be displayed incorrectly. ]
And the bit of the answer that Andreas left out was:
where you can download a Perl binding for DRMAA for Grid Engine.
Depending on how complex your objective is, DRMAA may be the best
choice. If you're needs are very simple, using the command-line
utilities ("qsub -sync y" and "qstat -j" and "qstat -xml" in particular)
may be the best approach. If you want to do anything that smells
multi-threaded or involves logic, DRMAA is the way to go.
Andreas.Haas at Sun.COM wrote:
> Hi Dan,
> On Fri, 14 Dec 2007, Dan McMahill wrote:
>> I've been reading various docs and am a bit overwhelmed. Before I go
>> further, I have a basic question about capability.
>> We have SGE deployed and I use qsub to submit jobs all the time.
>> qstat to check on jobs, and qdel to remove them from the queue. So
>> far the jobs have been fairly self contained. But now I'm interested
>> in writing a short program probably in perl (but ruby or maybe just
>> /bin/sh would be ok) that has a way of submitting jobs and monitoring
>> their status. Is this something SGE can easily do? If so, what
>> documents should I be reading to get going?
> this is what DRMAA API is for. It provides you operations for job
> submission, status control/monitoring plus a wait operation that
> mimics waitpid(2). Under
> you find the standard language independent interface specification.
> With regards to language bindings you can choose between C, Java,
> Perl, Python and Ruby. Online manuals for C and Java or links to
> scripting lanugage wrappers can be found under "Programming
> interfaces" of
> plus tutorials for C and Java under
>> If I were just writing my script for a multi-processor machine, I'd
>> just use fork/join to keep N jobs running at a time and monitor their
>> status with the parent process, but I'm not sure how to do that with
>> grid engine. In other words, I can make my program submit all of
>> these jobs to the queue, but I don't know how to automatically
>> monitor the results of the job short of some hacks where I have a
>> file in a shared file system that I monitor.
> Status monitoring can done with drmaa_job_ps(3)
> or drmaa_wait(3)
> if you just need to synchronize with job finishing.
> In case you are looking for a rather comprehensive DRMAA sample
> application that includes submission of new jobs upon jobs finishing
> flow.rb could be interesting
>> The application is running the same calculation (which is done via an
>> external program that may run for hours at a time) for different sets
>> of input parameters and after all jobs have completed, assembling the
>> results. Actually it would be neat if results were assembled as each
>> piece completed so I could get a partial picture along the way.
> Should be doable. Under
> there is a sample that illustrates how DRMAA can be used to
> implementing kind
> of a progress bar for compound jobs.
> Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551
> Amtsgericht Muenchen: HRB 161028
> Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
> Vorsitzender des Aufsichtsrates: Martin Haering
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users