[GE users] post task command

jpolk jimp at gradientfx.com
Thu Nov 13 19:44:42 GMT 2008


hmmm....let me research that a bit to see if it will work...

Ideally, what I need is a "post command" that once a job-task#
is finished it issue a "qalter" command, reassigning its machines
to another job-task#...

Thanks!
-Jim


reuti wrote:
> Am 13.11.2008 um 02:15 schrieb jpolk:
>
>   
>> Hi Reuti,....Thanks for your response....
>>
>> hmm,..I will search for more information on a queue "epilogue"....
>> (what a great term, btw)
>>
>> As far as our setup,...
>>
>>     
>>> Did I get you setup correct: you have e.g. 16 machines in pool A, 8
>>> in pool B and 4 in pool C, while jobs are only allowed during day to
>>> start in pool B or C. During night, after they are drained, you would
>>> like to have a setup with 28 nodes just in pool A?
>>>       
>> That's close....we have a simple setup really....but in the example
>> described, all the pools are available/dedicated for rendering,
>> so in the beginning, all pools are running concurrently.
>> Overnight, "B" and "C" run dry,etc...
>>     
>
> This sounds more like a setup for a calendar. In the general queue  
> you will have to define:
>
> $ qconf -sq pool_a.q
> ...
> calendar NONE,[@pool_b_hgrp=on_during_night], 
> [@pool_c_hgrp=on_during_night]
>
> For the two other queues:
>
> $ qconf -sq pool_b.q
> ...
> calendar off_during_night
>
> =====================
>
> calendars you can define with e.g. `qconf -acal on_during_night`, see  
> `man calendar_conf` for some examples. You could also name the other  
> on "on_during_day" if you prefer.
>
> =====================
>
> To avoid oversubscription of the nodes as there might still be some  
> jobs in pool B/C during the state change (until they are drained) you  
> will need either to limit the total number of slots per exechost in  
> "complex_values slots=8", or a resource quota (RQS) for it.
>
> -- Reuti
>
>
>   
>>> Are these 16+8+4 machines bound to dedicated nodes, or is it more
>>> like having this amount of slots for each job type?
>>>       
>> mmm, I believe the former....each of our rendering machines has 8  
>> cores,
>> split into 4 threads each, so two jobs can run simultaneously per  
>> machine
>> typically, though we can modify that if need be.
>>
>> Thanks again!
>> -Jim
>>
>>
>>
>>
>> reuti wrote:
>>     
>>> Hi Jim,
>>>
>>> Am 12.11.2008 um 22:06 schrieb jpolk:
>>>
>>>
>>>       
>>>> Just recently, we successfully implemented a strategy to allocate  
>>>> all
>>>> our rendering
>>>> machines into three "pools",...such that jobs from Project A only
>>>> run in
>>>> Pool A,
>>>> and so forth.  We do this by specifying a hostname list either at
>>>> submission time
>>>> with "qsub", or later with "qalter".
>>>>
>>>> We have a little conundrum however ;-)....
>>>>
>>>> During the wee hours of the night, the job(s, 'cause sometimes it's
>>>> only
>>>> one long job)
>>>> in Pool-B and Pool-C finish and so the machines in those pools go
>>>> idle.
>>>> We'd like to
>>>> have a way to automatically reallocate the machines from those idle
>>>> pools back into
>>>> Pool-A which is still running (in our example).
>>>>
>>>> I see that there is a way to send mail to users when a jobtask
>>>> finishes.
>>>> I wonder if there is a similar way to execute a command (probably a
>>>> "qalter" command)
>>>>
>>>>         
>>> in principle it could be done in a queue epilog, means to scan the
>>> waiting jobs and change their resource request. But I'm not sure,
>>> whether is the easiest approach.
>>>
>>>
>>>       
>>>> when a jobtask finishes which would make the adjustment.  I spent
>>>> sometime looking
>>>> thru the various syntax of "qalter", but I couldn't find anything  
>>>> that
>>>> appeared to do the trick.
>>>>
>>>> Does anybody have any experience with this? or might suggest another
>>>> alternative method?
>>>> Any and all input very much welcomed.
>>>>
>>>>         
>>> Did I get you setup correct: you have e.g. 16 machines in pool A, 8
>>> in pool B and 4 in pool C, while jobs are only allowed during day to
>>> start in pool B or C. During night, after they are drained, you would
>>> like to have a setup with 28 nodes just in pool A?
>>>
>>> Are these 16+8+4 machines bound to dedicated nodes, or is it more
>>> like having this amount of slots for each job type?
>>>
>>> -- Reuti
>>>
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do? 
>>> dsForumId=38&dsMessageId=88650
>>>
>>> To unsubscribe from this discussion, e-mail: [users- 
>>> unsubscribe at gridengine.sunsource.net].
>>>
>>>
>>>
>>>       
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do? 
>> dsForumId=38&dsMessageId=88654
>>
>> To unsubscribe from this discussion, e-mail: [users- 
>> unsubscribe at gridengine.sunsource.net].
>>     
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=88686
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>
>
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=88695

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list