[GE users] parallel environment, advanced reservation

reuti reuti at staff.uni-marburg.de
Wed Oct 27 00:09:09 BST 2010


Am 26.10.2010 um 02:27 schrieb peeter <phimmelf at hotmail.com>:

> Users submit single slot jobs that are spawning/forking cpu bound  
> threads.  In some cases eventually taking up 16 cores on a 24 core  
> host. Oftentimes the job will run for 5 days, yet only spawn/fork to  
> 16 threads(slots) for 10 hours of the entire 5 day run.
>
> I understand how to configure a pe and then have user pass, for  
> example, '-l pe smp 16' on the command line for the job described  
> above. But, obviously, that will take up all 16 slots for the entire  
> 5 days.
>
> I'm looking for techniques/strategies that would allow other jobs to  
> use the 15 cores when the job isn't spawning/forking. Some other  
> characteristics and comments to issue:
>
> 1. The spawning and forking cycles from 1 to 16 threads over the  
> course of 5 days
>
> 2. if advanced reservationing is used, what if user can't provide  
> exact times for when the job is increasing and decreasing thread  
> count for that single job?

There is nothing in SGE which provides this feature. It was just on  
the Open MPI mailing list where someone is developing such a  
scheduling system.


> 3. when advanced reservationing is used, and let's say a job makes a  
> reservation for 16 slots in a single host for 23 hours on Saturday,  
> do all other jobs submitted have to provide start and stop times so  
> that the

Normal jobs have no start time, but will be scheduled as soon as  
resources are available. The "-a" option to 'qsub' doesn't specify a  
start time, but a point in time from which on the job is eligible to  
be scheduled, but it might get scheduled later. When no "-l h_rt ..."  
is specified for normal jobs, the defined "default_duration" from the  
scheduler configuration will be taken to decide, whether a job still  
fits in the left timeframe regarding reservations (and calendars).

-- Reuti


> reservation system knows what jobs to prevent from running? For  
> instance, what if user submits a job Friday night to 20 slots on the  
> same host but doesn't know how long it's going to run, and doesn't  
> provide start/stop time? What will happen to the Friday night job if  
> it runs into Sunday?
>
> Forever indebted.
>
> Peeter
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=290080
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=290328

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list