[GE users] per (array) job epilog
Mark.Olesen at emconTechnologies.com
Tue Jun 30 09:52:47 BST 2009
> I have the feeling I'm missed something in the doku. I only found a
> to set an epilog for a queue. But I want to have a job specific
> (for cleanup even if the job failed or was qdel'ed). Is there such a
> thing or do I need to invent my own using the queue epilog and some
> clever mechanism?
Our prolog/epilog mechanism uses the requested resources to
differentiate between job types:
# (hard) requested resources
rclist=$($SGE_BINARY_PATH/qstat -r -j $JOB_ID | \
sed -ne 's/^.*hard *resource_list: */,/p')
case "$rclist," in
echo do something
echo do something else
Another possibility is to simply schedule a cleanup job that should run
immediately after the main job.
# step 1: submit our job ($0) and remember job id for the cleanup/kill
job_id=$($SGE_BINARY_PATH/qsub -o $logFile -terse $0 $@ 2>&1)
# check exitcode etc.
# step 2: submit cleanup/kill job
# cd HOME to avoid any weird open file-descriptors
# but provide spoolDir (or whatever) via job context
echo "schedule cleanup job"
( cd $HOME && qsub -N _$job_id -hold_jid $job_id -sc spoolDir=$spoolDir
To make sure the cleanup job runs immediately, you can give it a high
priority and/or send it to a special 'schedule' queue that only runs
these sorts of jobs. Since the load of these jobs is minimal, you can
just create a 'schedule' queue with many slots on your qmaster node.
This e-mail message and any attachments may contain
legally privileged, confidential or proprietary Information,
or information otherwise protected by law of EMCON
Technologies, its affiliates, or third parties. This notice
serves as marking of its "Confidential" status as defined
in any confidentiality agreements concerning the sender
and recipient. If you are not the intended recipient(s),
or the employee or agent responsible for delivery of this
message to the intended recipient(s), you are hereby
notified that any dissemination, distribution or copying
of this e-mail message is strictly prohibited.
If you have received this message in error, please
immediately notify the sender and delete this e-mail
message from your computer.
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users