[GE users] per (array) job epilog

olesen Mark.Olesen at emconTechnologies.com
Tue Jun 30 09:52:47 BST 2009


> I have the feeling I'm missed something in the doku. I only found a
> way
> to set an epilog for a queue. But I want to have a job specific
> epilog.
> (for cleanup even if the job failed or was qdel'ed). Is there such a
> thing or do I need to invent my own using the queue epilog and some
> clever mechanism?

Our prolog/epilog mechanism uses the requested resources to
differentiate between job types:

# (hard) requested resources
rclist=$($SGE_BINARY_PATH/qstat -r -j $JOB_ID | \
  sed -ne 's/^.*hard *resource_list: */,/p')


case "$rclist," in
*,abc=*)
    echo do something
    ;;
*,def=*)
    echo do something else
    ;;
esac


Another possibility is to simply schedule a cleanup job that should run
immediately after the main job. 

# step 1: submit our job ($0) and remember job id for the cleanup/kill
job_id=$($SGE_BINARY_PATH/qsub -o $logFile -terse $0 $@ 2>&1)
exitcode=$?
# check exitcode etc.
...
# step 2: submit cleanup/kill job
# cd HOME to avoid any weird open file-descriptors
# but provide spoolDir (or whatever) via job context
echo "schedule cleanup job"
( cd $HOME && qsub -N _$job_id -hold_jid $job_id -sc spoolDir=$spoolDir
$0-cleanup )


To make sure the cleanup job runs immediately, you can give it a high
priority and/or send it to a special 'schedule' queue that only runs
these sorts of jobs. Since the load of these jobs is minimal, you can
just create a 'schedule' queue with many slots on your qmaster node.


/mark
This e-mail message and any attachments may contain 
legally privileged, confidential or proprietary Information, 
or information otherwise protected by law of EMCON 
Technologies, its affiliates, or third parties. This notice 
serves as marking of its "Confidential" status as defined 
in any confidentiality agreements concerning the sender 
and recipient. If you are not the intended recipient(s), 
or the employee or agent responsible for delivery of this 
message to the intended recipient(s), you are hereby 
notified that any dissemination, distribution or copying 
of this e-mail message is strictly prohibited. 
If you have received this message in error, please 
immediately notify the sender and delete this e-mail 
message from your computer.

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=204555

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list