[GE users] Job submission speed through DRMAA api.
kruland at ku.edu
Fri Jan 21 16:07:01 GMT 2005
[ The following text is in the "ISO-8859-1" character set. ]
[ Your display is set for the "ISO-8859-10" character set. ]
[ Some special characters may be displayed incorrectly. ]
Please take these numbers as approximate. They are from a single run,
and include some console io, etc. This is a very rough breakdown of the
drmaa_init = 2 sec
drmaa_allocate_job = 0 sec (to 4 places)
drmaa_set_attribute ( script name ) = 0.04 sec
drmaa_run_job = 0.04 sec
drmaa_synchronize == not reporting since it doesn't mean much.
drmaa_exit = 1.0 sec.
Andreas is correct in that the drmaa_init time is negligable if
amortized over multiple jobs. But, suppose I know before hand that I
don't need to synchronize the jobs, would there be some way to forego
Thanks much for looking into this.
Andreas Haas wrote:
>Irrespective of possible inefficiencies in implementation you
>could add in case of DRMAA synchronization at session begin is
>always necessary to guarantee drmaa_wait() reliably works. For
>that reason a drmaa_init() delay is unavoidable even though
>I fully agree there should possibilities to lessen drmaa_init()
>Though I don't know the overall set-up used in that case but as
>long as the python script issues a number of drmaa_run_job()
>calls the session init delay should be negligible. Also the
>drmaa_init() overhead performance-wise will yield a sensible
>profit as soon as DRMAA capability to synchronize with jobs
>finish is utilized compared to qstat/qacct be used for that
>On Fri, 21 Jan 2005, Daniel Templeton wrote:
>>Duh. I should have known that. I wrote it. ;)
>>The reason for the difference is that qsub is single-threaded unless a
>>-sync or -now option is used. DRMAA is always mutli-threaded. At the
>>creation of the second thread, the second thread requests a current list
>>of running jobs from the qmaster. It and the main thread block until
>>that list is received.
>>OK, so we've accounted for 2 of 3.5 seconds. The other 1.5 must be in
>>the job templates and the drmaa_run_job() call. The next interesting
>>test would be to time how long the drmaa_run_job() call takes. I
>>suspect it's less than 0.1 seconds.
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users