[GE users] SGE-6.0u6: Limit the number of concurrently running array jobs?

Ron Chen ron_chen_123 at yahoo.com
Wed May 17 15:31:35 BST 2006


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

For per-job limit of max_aj_instances it is not currently
possible.

However, it shouldn't be very hard to do, we need to do:
1) enchance qsub to pass per-job max_aj_instances to qmaster
2) spool the value onto storage (classic/Berkeley DB)
3) in libs/sched/sge_job_schedd.c:split_jobs(), use the per-job
value to check whether array job's task_instances is exceeding
the limit.

(May be other developers can comment on whether this is enough
or not?)

BTW, since this feature has been requested for a few times
before, can Erik open an issue so that we can track this?

 -Ron


--- Andrea Lorenz <lorenz at rz.rwth-aachen.de> wrote:
> Hello Erik,
> 
> take a look at the manual page sge_conf
> 
> max_aj_instances
>        This parameter defines the maximum amount of array task
>  to  be  
> scheduled  to  run
>        simultaneously  per array job.  An instance of an array
> task will 
> be created within
>        the master daemeon when it gets a start order from the
> scheduler. 
> The instance will
>        be  destroyed  when  the  array task finishes.  Thus
> the 
> parameter provides control
>        mainly over the memory consumption of array jobs in the
> master 
> and  scheduler  dae-
>        mon.  It  is  most  useful  for very large clusters and
> very 
> large array jobs.  The
>        default for this parameter is 2000. The value 0 will
> deactivate 
> this limit and will
>        allow  the  scheduler  to  start  as many array job
> tasks as 
> suitable resources are
>        available in the cluster.
> 
> 
> Andrea
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail:
> users-help at gridengine.sunsource.net
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list