[GE users] Prolog

Andre Thomas athomas at forensicmedia.co.uk
Tue May 23 17:36:49 BST 2006


Excellent, thanks Dan, I'll give it try.

cheers

andre

Gruhn Daniel J Contractor AF/A9IT wrote:

>Andre,
>
>You can declare your eplilog command line with the variables specified in
>man sge_conf.  In this way, specify the job name and/or job ID (and any of
>the other available values) on the command line and then use them in your
>epilog to see if all of the tasks of your array job have completed.  If so,
>then the currenlty running epilog is the last one.
>
>You may need a little programming to guard against race conditions.  E.g. If
>the last two array tasks of your job finish at the same time, make sure only
>one runs.  Here is some code snippets that may help you.  You will have to
>figure out what globally accessable place you would put this status file.
>
>
>prolog
>------
># Get job ID from command line
>...
>
># Make status file name
>statusfile=status_$jobid
>
># If we can get a lock on the status file
>touch $statusfile
>if ln $statusfile $statusfile.lock
>then
>    # We have the lock, are we the first?
>    if [ -z "`cat $statusfile`" ]
>    then
>        # Yes we are the first, put in some status
>        echo "`date`: Running" >$statusfile
>    fi
>    rm $statusfile.lock
>fi
>
>
>epilog
>------
># Get job ID from command line
>...
>
># If the job still exists
>if qstat -j $jobid
>then
>    # We are not the last task epilog, just exit
>    exit 0
>fi
>
># Make status file name
>statusfile=status_$jobid
>
># If we can get a lock on the status file
>if ln $statusfile $statusfile.lock
>then
>    # Do the post-processing, which could include using the
>    # contents of the status file to compute time of the array
>    # job.
>    ...
>    rm -f $statusfile $statusfile.lock
>fi
>
>
>Good luck,
>
>Dan
> 
>//SIGNED//
>Daniel J.Gruhn, CTR (Group W Inc.)
>HQ USAF/A9IT
>Studies & Analyses, Assesments and Lessons Learned
>PH: DSN 425-6871 COM (703) 588-8671
>FX: DSN 426-8738 COM (703) 696-8738
>Daniel.Gruhn.ctr at pentagon.af.mil
>
>-----Original Message-----
>From: Andre Thomas [mailto:athomas at forensicmedia.co.uk] 
>Sent: Tuesday, May 23, 2006 11:24 AM
>To: users at gridengine.sunsource.net
>Subject: Re: [GE users] Prolog
>
>Dan,
>
>yes you are absolutely right, I did mean  epilog.
>
>Yes I am aware of creating another job dependant on the first, but if I can
>I like to avoid that, since we are generating over 400000 jobs a day and
>tracking all of them, and having another job run increases this by at least
>another 200k. So if there is a way it would be great.
>
>cheers
>
>andre
>
>Gruhn Daniel J Contractor AF/A9IT wrote:
>
>  
>
>>Andre,
>>
>>Just a wording correction, but would want an epilog (not a prolog).
>>
>>Another possibility is to queue another job which waits for the array 
>>job to finish before it runs.
>>
>>Dan
>>
>>
>>//SIGNED//
>>Daniel J.Gruhn, CTR (Group W Inc.)
>>HQ USAF/A9IT
>>Studies & Analyses, Assesments and Lessons Learned
>>
>>
>>-----Original Message-----
>>From: Andre Thomas [mailto:athomas at forensicmedia.co.uk]
>>Sent: Tuesday, May 23, 2006 11:13 AM
>>To: users at gridengine.sunsource.net
>>Subject: [GE users] Prolog
>>
>>A quick question, is it possible to have a prolog that runs only once 
>>when an entire array job is complete?
>>
>>Basically, I like to keep the job_id's down, since we generate and 
>>track some 400k per day, but need a check to run once an entire array 
>>job is finished.
>>
>>cheers
>>
>>andre
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>>
>>.
>>
>> 
>>
>>    
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
>
>
>  
>



More information about the gridengine-users mailing list