[GE users] Releasing requested consumable resources before job completion

Reuti reuti at staff.uni-marburg.de
Tue Sep 2 18:09:48 BST 2008


Am 01.09.2008 um 19:16 schrieb Stephen Siard:

> 2008/9/1 Reuti <reuti at staff.uni-marburg.de>
> Am 30.08.2008 um 19:52 schrieb Dioktos:
>
> On Sat, Aug 30, 2008 at 5:10 PM, Reuti <reuti at staff.uni-marburg.de>  
> wrote:
> Hi,
>
> Am 30.08.2008 um 16:45 schrieb Dioktos <dioktos at gmail.com>:
>
> I'm trying to find a way to effectively request the use of a CPU for a
> job and release it before the job terminates.
>
>
> Is there an effective workaround to this problem?
>
> did you try to use load_adjustments in the scheduler configuration  
> already?
>
> Ah- no. I missed that. I suppose that's generally what I'm looking for
> in this case. Thank you.
>
> ... After a bit of experimentation, I find that I need to use
> load_short (or something even faster) to get a decent response time.
>
> It's still not what I'm really after, though, because I want to be
> able to submit more low-CPU jobs and not have them bump the load
> adjustment.
>
> But you could bump up any other complex, which you e.g. assign only  
> to the jobs with the high resource request during the beginning.  
> Hence there are virtually less available. The bumped up complex can  
> then be used to put the queue for this type of jobs into "alarm  
> state", hence no further jobs of this type would go to this node,  
> until it's again above a certain value (how load_thresholds works  
> [triggered by "below" or "above"], depends on the relation defined  
> for a complex).
>
> Normal type jobs won't be effected this way, as they run in a  
> different queue and don't request thisd special resource.
>
> The scheduler "bump" for load adjustment isn't configurable for  
> different job types. So even if only the special jobs request it,  
> other jobs being submitted will still cause the load adjustment and  
> delay the execution of more "high resource" jobs.

Okay, then you could try a co-scheduler. A job can set a context  
(this has no meaning to SGE at all, it's just for user purpose) like:

$ qsub -sc extra_io=1 -h job.sh

The co-schduler could sweep the cluster for these type of jobs, and  
if it find a free node release one job of this type in the waiting  
state with `qrls <job_id>`. During the runtime of the job, it could  
itself change the value for this context:

$ qalter -sc extra_io=0 <job_id>

-- Reuti


> Basically, I want the number of high CPU jobs to be independent of  
> the number of low CPU jobs.
>
> Thanks,
> Stephen
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list