[GE users] release a resource in a middle of a job by the job itself

lily lily.li at pgs.com
Wed Jul 14 16:58:32 BST 2010


    [ The following text is in the "utf-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hi, Reuti,

We have similar need to release resources (actually the slots) in the middle of the running jobs. But the job is an MPI job, so we can't split it into multiple jobs. 

For a job that has load balancing problem ( depending on input data), some MPI ranks finish early, but have to wait for the whole job to come down, and thus wasting a lots of node hours. Is there a way to release the slots occupied by the early finished MPI ranks to qmaster/scheduler?

We are advised to increase the slots number configured for the node when an MPI rank finishes its work and goes to MPI_Finalize(), is it safe to do so?

Regards,
Lily

-----Original Message-----
From: reuti [mailto:reuti at staff.uni-marburg.de] 
Sent: Tuesday, June 15, 2010 12:47 PM
To: users at gridengine.sunsource.net
Subject: Re: [GE users] release a resource in a middle of a job by the job itself

Hi,

Am 15.06.2010 um 15:39 schrieb introx:

> I could do so, the problem is that I don't know in advance how many parts will be in this chain...

maybe you can put this in two array jobs:

-- For job_a.sh you set -tc 1 (only one running at a time) and an range by -t which will most likely cover all steps in the worst case and requesting a license.

-- For job_b.sh you set a -hold_jid_ad with the same -t range (waiting for each steps finish). When this step doesn't need a license, even more than one could start at a time (they are limited by job_a.sh anayway).

-- Reuti


> On Tue, Jun 15, 2010 at 4:26 PM, dr_st <stephane.teletchea at univ-nantes.fr> wrote:
> introx a écrit :
> > Hi,
> >
> > I have a job which uses a licenses but it doesn't need the license all
> > the time so it can release it when it doesn't use it.
> >
> > Can a job release a resource (such as license resource...) in the middle
> > of its operation?
> > And if so can it acquire this resource later on?
> >
> >
> > Thanks
> > Erez
> >
> 
> You could try to split your main job:
> - job_A.sh launches the program with license, and when complete launches
> - job_B.sh which performs the rest of the analysis.
> 
> Chaining jobs is probably a good option there, no?
> 
> Cheers,
> Stéphane
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=262127
> 
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=262199

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

This e-mail, including any attachments and response string, may contain proprietary information which is confidential and may be legally privileged. It is for the intended recipient only. If you are not the intended recipient or transmission error has misdirected this e-mail, please notify the author by return e-mail and delete this message and any attachment immediately. If you are not the intended recipient you must not use, disclose, distribute, forward, copy, print or rely on this e-mail in any way except as permitted by the author.

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=268000

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list