[GE users] Job submission speed through DRMAA api.

Kevin Ruland kruland at ku.edu
Thu Jan 20 12:47:02 GMT 2005


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]


All,

Ron:  I understand the difference you are looking for.  I am calling 
submission speed the time from drmaa_init, to the return from drmaa_run_job.

Alas there appears to be a significant difference between qsub and drmaa 
in job submission.  I haven't tried to dermine if there is a difference 
in dispatching.

 From the command line I'm doing this:

qsub -p 0 -b yes -shell no -w e sleeper.sh

the command line time tool tells me it returns in < 0.1 second.

I then compiled howto2.c and its complete run too about 6 seconds.  
Recall that howto.c does the following:
  drmaa_init
  drmaa_allocate_job_template
  drmaa_set_attribute - remote command
  drmaa_run_job
  drmaa_delete_job_template
  drmaa_exit

This takes between 4 and 6 seconds to execute.

I notices that some time happened between the printf with the jobid and 
the program termination, so I cut out the delete_job_template and exit 
calls and the time is at about 3.5seconds.

I've the copy of the c source which doesn't shut down drmaa gracefully.

Kevin

Kevin Ruland wrote:

>
> All,
>
> My situation is a little more complex than I first let on. First I'm 
> not using DRMAA directly, but rather through python swig wrappers.  So 
> I need to figure out what I'm doing in there, recode in straight C and 
> see if there is any significant difference.  So I don't have a nice 
> test case yet.
>
> I also don't have hard numbers.  It's all a feeling now.
>
> Kevin
>
> Ron Chen wrote:
>
>> Kevin,
>>
>> Do you have a testcase?
>>
>> Also, is it the submission speed that is slow, or the
>> job dispatch speed that is slow? They are very
>> different, so may be you can tell us more about how
>> you discovered the speed problem.
>>
>> -Ron
>>
>>
>> --- Stephan Grell <stephan.grell at sun.com> wrote:
>>  
>
>


    [ Part 2: "Attached Text" ]

#include <stdio.h>
#include "drmaa.h"

int main (int argc, char **argv) {
  char error[DRMAA_ERROR_STRING_BUFFER];
  int errnum = 0;
  drmaa_job_template_t *jt = NULL;
  
  errnum = drmaa_init (NULL, error, DRMAA_ERROR_STRING_BUFFER);

  if (errnum != DRMAA_ERRNO_SUCCESS) {
    fprintf (stderr, "Could not initialize the DRMAA library: %s\n", error);
    return 1;
  }

  errnum = drmaa_allocate_job_template (&jt, error, DRMAA_ERROR_STRING_BUFFER);

  if (errnum != DRMAA_ERRNO_SUCCESS) {
    fprintf (stderr, "Could not create job template: %s\n", error);
    return 1;
  }

  errnum = drmaa_set_attribute (jt, DRMAA_REMOTE_COMMAND, "sleeper.sh",
				error, DRMAA_ERROR_STRING_BUFFER);

  if (errnum != DRMAA_ERRNO_SUCCESS) {
    fprintf (stderr, "Could not set attribute \"%s\": %s\n",
	     DRMAA_REMOTE_COMMAND, error);
    return 1;
  }

  char jobid[DRMAA_JOBNAME_BUFFER];

  errnum = drmaa_run_job (jobid, DRMAA_JOBNAME_BUFFER, jt, error,
			  DRMAA_ERROR_STRING_BUFFER);

  if (errnum != DRMAA_ERRNO_SUCCESS) {
    fprintf (stderr, "Could not submit job: %s\n", error);
    return 1;
  }
  printf ("Your job has been submitted with id %s\n", jobid);

  return 0;
}



    [ Part 3: "Attached Text" ]

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net



More information about the gridengine-users mailing list