[GE users] Job submission speed through DRMAA api.

Daniel Templeton Dan.Templeton at Sun.COM
Fri Jan 21 16:28:11 GMT 2005


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

No, unfortunately not.  I suppose it would be possible to build a switch
into the contact string (when we get around to building the contact
string), something like "ROOT=/sge;CELL=default;THREADS=single".  It
would be similar to JDBC database URLs.  Andreas, what do you think
about that?

Daniel

Kevin Ruland wrote:

>Please take these numbers as approximate.  They are from a single run, 
>and include some console io, etc.  This is a very rough breakdown of the 
>times:
>
>drmaa_init = 2 sec
>drmaa_allocate_job = 0 sec (to 4 places)
>drmaa_set_attribute ( script name ) = 0.04 sec
>drmaa_run_job = 0.04 sec
>drmaa_synchronize == not reporting since it doesn't mean much.
>drmaa_exit = 1.0 sec.
>
>Andreas is correct in that the drmaa_init time is negligable if 
>amortized over multiple jobs.  But, suppose I know before hand that I 
>don't need to synchronize the jobs, would there be some way to forego 
>the init?
>
>Thanks much for looking into this.
>
>Kevin
>
>Andreas Haas wrote:
>
>  
>
>>Irrespective of possible inefficiencies in implementation you
>>could add in case of DRMAA synchronization at session begin is
>>always necessary to guarantee drmaa_wait() reliably works. For
>>that reason a drmaa_init() delay is unavoidable even though
>>I fully agree there should possibilities to lessen drmaa_init()
>>delay.
>>
>>Though I don't know the overall set-up used in that case but as
>>long as the python script issues a number of drmaa_run_job()
>>calls the session init delay should be negligible. Also the
>>drmaa_init() overhead performance-wise will yield a sensible
>>profit as soon as DRMAA capability to synchronize with jobs
>>finish is utilized compared to qstat/qacct be used for that
>>purpose.
>>
>>HTH,
>>Andreas
>>
>>On Fri, 21 Jan 2005, Daniel Templeton wrote:
>>
>> 
>>
>>    
>>
>>>Duh.  I should have known that.  I wrote it. ;)
>>>
>>>The reason for the difference is that qsub is single-threaded unless a
>>>-sync or -now option is used.  DRMAA is always mutli-threaded.  At the
>>>creation of the second thread, the second thread requests a current list
>>>of running jobs from the qmaster.  It and the main thread block until
>>>that list is received.
>>>
>>>OK, so we've accounted for 2 of 3.5 seconds.  The other 1.5 must be in
>>>the job templates and the drmaa_run_job() call.  The next interesting
>>>test would be to time how long the drmaa_run_job() call takes.  I
>>>suspect it's less than 0.1 seconds.
>>>
>>>Daniel
>>>
>>>   
>>>
>>>      
>>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>  
>

-- 
***************************************************
*        Daniel Templeton   ERGB01 x60220         *
*       Staff Engineer, Sun N1 Grid Engine        *
***************************************************
* "Roads? Where we're going we don't need roads." *
*                    -Dr. Emmett Brown            *
*                     Back to the Future (1985)   *
***************************************************



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list