[GE users] Creating a Soft consumable resource

reuti reuti at staff.uni-marburg.de
Tue Oct 19 12:27:09 BST 2010


Hi,

Am 19.10.2010 um 05:23 schrieb stephendennis:

> This thread fizzled out in July.  Does anyone know why soft consumables are
> denied, and if there is some way around this?

how do you expect soft-consumables to work? For ulimit like complexes like h_rt it's (the s_rt) like a "lower limit", but for boolean it's only like: I prefer a machine/queue with feature XYZ but I'm also happy with anything else.

- So for soft consumables it could mean either: I want at least (soft limit), but up to the max (hard limit).

or

- I need N of this soft complex, but I'm also happy if I don't get any at all.

Both would imply to get the information about the granted consumable complex into the job script to behave accordingly to the granted resources.

-- Reuti


> Thanks
> Stephen
> 
> ===========================================
> 
> 
> I went through the source code and the code to deny soft resources consumables
> was added in SGE v5.3 to fix the bug id
> http://gridengine.sunsource.net/issues/show_bug.cgi?id=223
> 
> http://gridengine.sunsource.net/ds/viewMessage.do?dsMessageId=78898&dsForumId=35
> 
> Quoting: ( 
> <> 
> I think there is a little bug inside SGE 5.3 .  
> If I have a host complex 
> mpi_down mpi_down BOOL false == NO NO false 
> and a queue with 
> load_thresholds      np_load_short=1.1,mpi_down=true 
> this queues is alarmed when no loadreport set mpi_down 
> to false.  
> </>
> 
> From: ./daemons/qmaster/sge_job_qmaster.c
> <>
> /****** sge_job/deny_soft_consumables() ****************************************
> *  NAME
> *     deny_soft_consumables() -- Deny soft consumables
> *
> *  SYNOPSIS
> *     static int deny_soft_consumables(lList **alpp, lList *srl)
> *
> *  FUNCTION
> *     Find out if consumables are requested and deny them.
> *
> *  INPUTS
> *     lList** alpp                    - answer list pointer pointer
> *     lList *srl                      - jobs JB_soft_resource_list
> *     const lList *master_centry_list - the master centry list
> *
> *  RESULT
> *     static int - 0 request can pass
> *                !=0 consumables requested soft
> *
> *******************************************************************************/
> int deny_soft_consumables(lList **alpp, lList *srl, const lList
> *master_centry_list)
> {
>   lListElem *entry, *dcep;
>   const char *name;
> 
>   DENTER(TOP_LAYER, "deny_soft_consumables");
> 
>   /* ensure no consumables are requested in JB_soft_resource_list */
>   for_each(entry, srl) {
>      name = lGetString(entry, CE_name);
> 
>      if (!(dcep = centry_list_locate(master_centry_list, name))) {
>         ERROR((SGE_EVENT, MSG_ATTRIB_MISSINGATTRIBUTEXINCOMPLEXES_S , name));
>         answer_list_add(alpp, SGE_EVENT, STATUS_EUNKNOWN,
> ANSWER_QUALITY_ERROR);
>         DRETURN(-1);
>      }
> 
>      /* ignore non-consumables */
>      if (lGetUlong(dcep, CE_consumable)) {
>         ERROR((SGE_EVENT, MSG_JOB_MOD_SOFTREQCONSUMABLE_S, name));
>         answer_list_add(alpp, SGE_EVENT, STATUS_EUNKNOWN,
> ANSWER_QUALITY_ERROR);
>         DRETURN(-1);
>      }
>   }
> 
>   DRETURN(0);
> }
> </>
> 
> <>
> 
>            if (deny_soft_consumables(alpp, lGetList(jep,
> JB_soft_resource_list), master_centry_list)) {
>            DRETURN(STATUS_EUNKNOWN);
>         }
> </>
> 
> Can someone more knowledgeable please comment on the changes or point us to some
> documentation on the soft resources.
> 
> Thanks.
> 
> -----Original Message-----
> From: Molumuri, Janardhan 
> Sent: Monday, July 19, 2010 8:02 PM
> To: use... at gridengine.sunsource.net
> Cc: svibhore
> Subject: RE: [GE users] Creating a Soft consumable resource
> 
> A soft request on consumable attributes is not working.
> 
> From the source code:
> 
> From msg_qmaster.h:
> <<>>
> #define MSG_JOB_MOD_SOFTREQCONSUMABLE_S  _MESSAGE(33307, _("denied: soft
> requests on consumables like "SFQ" are not supported"))
> <</>>
> 
> Is there any reason behind enforcing this? If so how can we use soft requests.
> 
> qsub -soft -l test=0.1 uptime
> Unable to run job: denied: soft requests on consumables like "test" are not
> supported.
> Exiting.
> 
> -----Original Message-----
> From: templedf [mailto:dani... at oracle.com] 
> Sent: Wednesday, July 14, 2010 7:55 PM
> To: use... at gridengine.sunsource.net
> Cc: svibhore
> Subject: Re: [GE users] Creating a Soft consumable resource
> 
> The qsub -soft switch with a consumable is what you want.  If you assign 
> 1 of your consumable to every machine and then request it with -soft, 
> the scheduler will try to put only one per machine, but if it runs out 
> of machines, it will ignore the consumable.
> 
> Daniel
> 
> On 07/14/10 04:33 AM, svibhore wrote:
> I have a scenario where I want to create a resource which should be used
> ( ideally) one per host but if all the resources are used( i.e all the
> hosts have job using this consumable) the jobs should not fail and get
> scheduled on any host.
> 
> In a typical hard resource the jobs would fail if all the hosts have one
> job running on them, and in soft resource I cannot limit the number of
> jobs getting scheduled on the host( which will be limited by number of
> slots).
> 
> This will achieve two things for me, first the jobs gets distributed on
> all host as one job per host and secondly this will guarantee that jobs
> would not fail even if all the hosts have a job running on them.
> 
> Ideas ?
> 
> Regards,
> Shantanu
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=267984
> 
> To unsubscribe from this discussion, e-mail:
> [user... at gridengine.sunsource.net].
> 
> 
> 
> ---------------------------------------------------------------------
> 
> 
> Notice from Univa UD Postmaster:
> 
> 
> This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. This message has been content scanned by the Univa UD Tumbleweed MailGate.
> 
> 
> 
> ---------------------------------------------------------------------
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=288284
> 
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=288343

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list