[GE users] Releasing requested consumable resources before job completion
reuti at staff.uni-marburg.de
Tue Sep 2 18:09:48 BST 2008
Am 01.09.2008 um 19:16 schrieb Stephen Siard:
> 2008/9/1 Reuti <reuti at staff.uni-marburg.de>
> Am 30.08.2008 um 19:52 schrieb Dioktos:
> On Sat, Aug 30, 2008 at 5:10 PM, Reuti <reuti at staff.uni-marburg.de>
> Am 30.08.2008 um 16:45 schrieb Dioktos <dioktos at gmail.com>:
> I'm trying to find a way to effectively request the use of a CPU for a
> job and release it before the job terminates.
> Is there an effective workaround to this problem?
> did you try to use load_adjustments in the scheduler configuration
> Ah- no. I missed that. I suppose that's generally what I'm looking for
> in this case. Thank you.
> ... After a bit of experimentation, I find that I need to use
> load_short (or something even faster) to get a decent response time.
> It's still not what I'm really after, though, because I want to be
> able to submit more low-CPU jobs and not have them bump the load
> But you could bump up any other complex, which you e.g. assign only
> to the jobs with the high resource request during the beginning.
> Hence there are virtually less available. The bumped up complex can
> then be used to put the queue for this type of jobs into "alarm
> state", hence no further jobs of this type would go to this node,
> until it's again above a certain value (how load_thresholds works
> [triggered by "below" or "above"], depends on the relation defined
> for a complex).
> Normal type jobs won't be effected this way, as they run in a
> different queue and don't request thisd special resource.
> The scheduler "bump" for load adjustment isn't configurable for
> different job types. So even if only the special jobs request it,
> other jobs being submitted will still cause the load adjustment and
> delay the execution of more "high resource" jobs.
Okay, then you could try a co-scheduler. A job can set a context
(this has no meaning to SGE at all, it's just for user purpose) like:
$ qsub -sc extra_io=1 -h job.sh
The co-schduler could sweep the cluster for these type of jobs, and
if it find a free node release one job of this type in the waiting
state with `qrls <job_id>`. During the runtime of the job, it could
itself change the value for this context:
$ qalter -sc extra_io=0 <job_id>
> Basically, I want the number of high CPU jobs to be independent of
> the number of low CPU jobs.
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users