[GE users] Prolog

Gruhn Daniel J Contractor AF/A9IT Daniel.Gruhn.ctr at pentagon.af.mil
Tue May 23 17:32:07 BST 2006


Andre,

You can declare your eplilog command line with the variables specified in
man sge_conf.  In this way, specify the job name and/or job ID (and any of
the other available values) on the command line and then use them in your
epilog to see if all of the tasks of your array job have completed.  If so,
then the currenlty running epilog is the last one.

You may need a little programming to guard against race conditions.  E.g. If
the last two array tasks of your job finish at the same time, make sure only
one runs.  Here is some code snippets that may help you.  You will have to
figure out what globally accessable place you would put this status file.


prolog
------
# Get job ID from command line
...

# Make status file name
statusfile=status_$jobid

# If we can get a lock on the status file
touch $statusfile
if ln $statusfile $statusfile.lock
then
    # We have the lock, are we the first?
    if [ -z "`cat $statusfile`" ]
    then
        # Yes we are the first, put in some status
        echo "`date`: Running" >$statusfile
    fi
    rm $statusfile.lock
fi


epilog
------
# Get job ID from command line
...

# If the job still exists
if qstat -j $jobid
then
    # We are not the last task epilog, just exit
    exit 0
fi

# Make status file name
statusfile=status_$jobid

# If we can get a lock on the status file
if ln $statusfile $statusfile.lock
then
    # Do the post-processing, which could include using the
    # contents of the status file to compute time of the array
    # job.
    ...
    rm -f $statusfile $statusfile.lock
fi


Good luck,

Dan
 
//SIGNED//
Daniel J.Gruhn, CTR (Group W Inc.)
HQ USAF/A9IT
Studies & Analyses, Assesments and Lessons Learned
PH: DSN 425-6871 COM (703) 588-8671
FX: DSN 426-8738 COM (703) 696-8738
Daniel.Gruhn.ctr at pentagon.af.mil

-----Original Message-----
From: Andre Thomas [mailto:athomas at forensicmedia.co.uk] 
Sent: Tuesday, May 23, 2006 11:24 AM
To: users at gridengine.sunsource.net
Subject: Re: [GE users] Prolog

Dan,

yes you are absolutely right, I did mean  epilog.

Yes I am aware of creating another job dependant on the first, but if I can
I like to avoid that, since we are generating over 400000 jobs a day and
tracking all of them, and having another job run increases this by at least
another 200k. So if there is a way it would be great.

cheers

andre

Gruhn Daniel J Contractor AF/A9IT wrote:

>Andre,
>
>Just a wording correction, but would want an epilog (not a prolog).
>
>Another possibility is to queue another job which waits for the array 
>job to finish before it runs.
>
>Dan
>
>
>//SIGNED//
>Daniel J.Gruhn, CTR (Group W Inc.)
>HQ USAF/A9IT
>Studies & Analyses, Assesments and Lessons Learned
>
>
>-----Original Message-----
>From: Andre Thomas [mailto:athomas at forensicmedia.co.uk]
>Sent: Tuesday, May 23, 2006 11:13 AM
>To: users at gridengine.sunsource.net
>Subject: [GE users] Prolog
>
>A quick question, is it possible to have a prolog that runs only once 
>when an entire array job is complete?
>
>Basically, I like to keep the job_id's down, since we generate and 
>track some 400k per day, but need a check to run once an entire array 
>job is finished.
>
>cheers
>
>andre
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
>
>.
>
>  
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list