[GE users] Dynamically changing the slots value in the queue

reuti reuti at staff.uni-marburg.de
Thu Nov 25 09:30:20 GMT 2010


Well, using colors was indeed better to read (although I hate formatted emails under normal circumstances), but not easier to write. Strange - is anyone else using Yahoo and noticed their strange formatting too?


Am 24.11.2010 um 13:17 schrieb llikethat:

> Why is it so hard to read your replies - they are always quoted as being part of the original message. It's coming from Yahoo AFAICS - is it their feature?
> 
> I think it is a Yahoo feature, not sure. but now im not specifying the quote mentioned in the threads.
> 
> Why? What is the documentation of your application saying. In the worst case some can be forced by using the mentioned variable to run in serial. If it's only using threads from time to time you are out of luck anyway. In this case it would be best to force all to run serial but run four of them at a time. In total you lose less computing time.
> 
> I'm sorry i didn't understand this. If i'm using all the slots to run the job how will i lose on the computing time.

You stated, that you don't know beforehand how many cores the application will use. So I assume, that the application will run only for certain steps in parallel, but other steps in serial. So the amount of used cores varies over the lifetime of a job. But this can't be predicted or used by SGE.

When you instead force the application to run serial all the time, you could start four of them at once, and all are using their granted core all the time to 100%.


> No, migration between queues is not implemented.
> 
> NB: You could setup and abuse the checkpointing environemnt for this, but I don't think it would be a good solution.
> 
> I tried reading about checkpointing - but the application we use does not support checkpointing. I was also not sure how to implement or make it work for a 3d animation package.

Yes, checkpointing would have to be provided by the application or on a kernel level. You could force the job to run in another queue, but it would start otherwise from the beginning.

-- Reuti


> Thanks,
> 
> --- On Wed, 24/11/10, reuti <reuti at staff.uni-marburg.de> wrote:
> 
> From: reuti <reuti at staff.uni-marburg.de>
> Subject: Re: [GE users] Dynamically changing the slots value in the queue
> To: users at gridengine.sunsource.net
> Date: Wednesday, 24 November, 2010, 5:15 PM
> 
> Am 24.11.2010 um 12:13 schrieb llikethat:
> 
> > --- On Wed, 24/11/10, reuti <reuti at staff.uni-marburg.de> wrote:
> > 
> > From: reuti <reuti at staff.uni-marburg.de>
> > Subject: Re: [GE users] Dynamically changing the slots value in the queue
> > To: users at gridengine.sunsource.net
> > Date: Wednesday, 24 November, 2010, 2:15 PM
> > 
> > Am 24.11.2010 um 06:03 schrieb llikethat:
> > 
> > > --- On Tue, 23/11/10, reuti <reuti at staff.uni-marburg.de> wrote:
> > > 
> > > From: reuti <reuti at staff.uni-marburg.de>
> > > Subject: Re: [GE users] Dynamically changing the slots value in the queue
> > > To: users at gridengine.sunsource.net
> > > Date: Tuesday, 23 November, 2010, 5:23 PM
> > > 
> > > Hi,
> > > 
> > > Am 23.11.2010 um 12:31 schrieb llikethat:
> > > 
> > > > When i'm running a certain job, the cpu usage is 100% the slots value for this node's queue is 1 by default. Now if i'm running a job which uses only 20% of the cpu is possible to dynamically change the slots value to 5 so that the job uses the maximum available resources?
> > > 
> > > I would suggest to investigate, why this job isn't running at 100%.
> > > 
> > > 20% which i mentioned was just a rough number, but the actual is like this, certain rendering process does not use all the cores in the machine like if i have 4 cores it uses only 1 core leaving the rest 3 cores free. The idea is if i can identify this usage at runtime then i can start multiple threads depending on the resource availability.
> > 
> > Then the best is to define the slots for a queue equal to the installed cores and submit a serial job as such when they use only one core, other jobs which use more than one core have to be submitted as parallel ones. Each of the parallel jobs should use exactly the granted amount of cores. Whether they use forks or threads.
> > 
> > For OpenMP you can set e.g. in your jobscript: export OMP_NUM_THREADS=$NSLOTS
> > 
> > -- Reuti
> > 
> > Thank you for the reply. But we are not working on parallel jobs at all. These are 3d animation renders, so we are using array jobs for submission to the grid.
> 
> Why is it so hard to read your replies - they are always quoted as being part of the original message. It's coming from Yahoo AFAICS - is it their feature?
> 
> 
> > As you said the better option would be to create different queues with different slots value and submit accordingly. But before starting a job in SGE we will not know if the job is using 1 core or all the 4 cores.
> 
> Why? What is the documentation of your application saying. In the worst case some can be forced by using the mentioned variable to run in serial. If it's only using threads from time to time you are out of luck anyway. In this case it would be best to force all to run serial but run four of them at a time. In total you lose less computing time.
> 
> 
> > How do i automate the following,
> > 
> > For instance, if i have 2 queues (a) slot4.q (b) slot1.q
> > 
> > if i submit a job to slot1.q and realise that the job is not utilizing all the core for computing, then what would be the better way to migrate the job to slot4.q or vice versa
> 
> No, migration between queues is not implemented.
> 
> NB: You could setup and abuse the checkpointing environemnt for this, but I don't think it would be a good solution.
> 
> 
> > If i can get an approach to this then i can create multiple queues with different slot combination and migrate them automatically after knowing the cpu usage.
> > 
> > 
> > 
> > ------------------------------------------------------
> > http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=298261
> > 
> > To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
> >
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=298296
> 
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=298658

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list