[GE users] job submission verifier

Andy Schwierskott andy.schwierskott at sun.com
Tue Sep 23 13:17:57 BST 2008


Hi,

I would like to add a couple of comments to our suggestion of the job
sumbission verifier (JSV):

- When you look at the functional specification you will see that we did not
  choose to fork off a script like other DRM systems do. The main reason for
  it is performance when the submission verfier is executed in qmaster. When
  qmaster needs to fork for every job sumbission a script it would have a
  serious impact on the qmaster performance. The stdin/out solution should be
  somewhat faster. It's still not ideal in terms of performance and will
  impact submit performance. A couple of future extension are possible:

    - support loading of a shared library provided by the admin
    - develop a simple language directly interpreted in qmaster which does
      the necessary verifications and adjustments to the job
    - may be even implement this language in Java and let the Java thread n
      qmaster do this work (this certainly would required addtional changes
      in qmaster to make this happen)

   I think the 'language' approach in qmaster is the only versatile approach
   which would allow to do job submissions with negligible impact and
   submission performance. It's probably a nice candidate for a community
   contribution??

   What is your experiences with the speed of script languages? How do shell
   languages compare to Perl and other interpreter languages? Your
   contributions in oyur favorite script langueage to provide sample or
   generic job submission verifier scripts to the next SGE release are
   welcome!

- under (N1) you see that the current proposal also includes DRMAA for
  the client side verification. We believe this would not require any
  changes to the DRMAA standard, but there could be unforseen side effects
  (it's similar to the fact that DRMAA reads the "sge_request" file). Please
  let us know you opinion on it.

- The client side verification always can be circumvented by a clever user -
  so it should be rather seen is a courtesy for the end users to ensure
  theirs job is runnable. The qmaster side verification is where an admin
  enforces that users request onlyaccepted resources

Andy

> Hi,
>
> in the past some users expressed their need for some kind of presubmission
> procedure which is executed whenever a job enters the GE system (see also
> issue #2621).
>
> Find attached a draft for a corresponding GE enhancement. Please give feedback
> by Friday.
>
> Regards,
>
> Ernst

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list