[GE users] Querying available slots with a resource string

Brad Falch bradleyfalch at yahoo.com
Thu Nov 8 23:03:34 GMT 2007


Daniel,

I figured out how to do it by using qstat instead:

qselect -l os_version=WS4.0,amd64=1 | xargs qstat -F -q | awk -F= ' /^.*qc:slots/ { sum+=$2 } END { print sum } '
However, I now have another problem.  We use projects to call out some hosts for interactive and some for batch sessions.  These jobs will go to batch sessions, so I need a way to distinguish between them.  Any ideas there?  (Looks like -P is not available to qselect or qstat, only the submission and admin commands.)

Thanks,

Brad

----- Original Message ----
From: Daniel Templeton <Dan.Templeton at Sun.COM>
To: users at gridengine.sunsource.net
Sent: Thursday, November 8, 2007 4:38:21 PM
Subject: Re: [GE users] Querying available slots with a resource string

Brad,

Duh.  Reuti pointed that out to me, but I hadn't had enough coffee yet 
to comprehend what he meant.  Yes.  Ignore my previous "helpful"
 advice.

qstat -f -l resource=value will show you the queues (including 
free/total slots) with those resources, as well as all jobs running in 
those queues and all pending jobs.  You'll have to parse out and sum up
 
the free slot count.  The -xml switch might make the output easier for 
you to parse.

I can't come up with a simpler solution off the top of my head.

Daniel

Brad Falch wrote:
> Daniel,
>
> Looks like the one-line script only shows the full number of slots
 that fit that description, not the slots available for running.  Is that
 correct?  I wanted to see how many are open.
>
> Thanks,
>
> Brad
>
> ----- Original Message ----
> From: Daniel Templeton <Dan.Templeton at Sun.COM>
> To: users at gridengine.sunsource.net
> Sent: Thursday, November 8, 2007 3:28:41 PM
> Subject: Re: [GE users] Querying available slots with a resource
 string
>
> Brad,
>
> Reuti sent me a cleaner way to write the script:
>
> qselect -l resource=value | xargs qconf -sq | awk ' /^slots/ {
 sum+=$2
>  } 
> END { print sum } '
>
> Daniel
>
> Brad Falch wrote:
>   
>> Daniel,
>>
>> Thanks for explaining and the shell script.   I'll give it a shot.
>>
>> -Brad
>>
>> ----- Original Message ----
>> From: Daniel Templeton <Dan.Templeton at Sun.COM>
>> To: users at gridengine.sunsource.net
>> Sent: Thursday, November 8, 2007 1:28:08 PM
>> Subject: Re: [GE users] Querying available slots with a resource
>>     
>  string
>   
>> Brad,
>>
>> There are two answers.  The first is to use the PE functionality
 that
>>     
>  
>   
>> SGE provides.  I assume you're talking about a parallel job since
>>  you've 
>> got a single job that's using multiple CPUs simultaneously.  (I know
>>  you 
>> know some of this already, but bear with me.)  When you request a PE
>>     
>  at
>   
>>  
>> job submission time, you can request a number of slots (-pe name
>>     
>  slots,
>   
>>  
>> e.g. -pe make 4).  The job will be accepted, but it will not be 
>> scheduled until the requested number of slots is available.  To
>>     
>  prevent
>   
>>  
>> the job from being accepted, you can use the "-now y" switch to
>>     
>  request
>   
>>  
>> that if the job cannot be immediately scheduled, that the job
>>  submission 
>> be rejected.  You also have the ability to request a slot range
>>     
>  instead
>   
>>  
>> of a fixed number (-pe name min-max, e.g. -pe make 4-16).  When 
>> requesting a slot range, the scheduler will give you the largest
>>     
>  number
>   
>>  
>> of slots in that range that is possible.  By submitting a slot range
 
>> with -now, you let the scheduler assign you as many slots as it can,
>>  and 
>> if it can't meet your minimum requirements, the submission will be
>>  rejected.
>>
>> The second answer is qselect and some shell scripting, something
>>     
>  like:
>   
>> #!/bin/sh
>> sum=0
>> for q in `qselect -l $1=$2`
>> do
>>    tmp=`qconf -sq $q | grep slots | awk '{ print $2 }'`
>>    sum=`expr $sum + $tmp`
>> done
>> echo $sum
>>
>> Daniel
>>
>> Brad Falch wrote:
>>   
>>     
>>> Hello, All,
>>>
>>> Please forgive this newbie question.  From the user perspective, is
>>>     
>>>       
>>  there a way to report the number of slots available on a Grid,
 given
>>     
>  a
>   
>>  particular resource string?  I'm hoping there is something built
>>     
>  into
>   
>>  SGE, but any suggestions for scripts are welcome.
>>   
>>     
>>> I run distributed processing jobs on a cluster that is
>>>       
>  heterogeneous,
>   
>>>     
>>>       
>>  and I only want to launch the job if a certain number of slots (all
>>  requested) are available.  I would also like to use this command
>>  interactively to see what is available and decide how many slots to
>>     
>  use.  (I
>   
>>  also don't want to wait for the job to start.  I do understand
 there
>>     
>  is a
>   
>>  way to submit so it only starts once all slots are available.)
>>   
>>     
>>> Thanks,
>>>
>>> Brad
>>>
>>>
>>>
>>>
>>>
>>> __________________________________________________
>>> Do You Yahoo!?
>>> Tired of spam?  Yahoo! Mail has the best spam protection around 
>>> http://mail.yahoo.com 
>>>
>>>
>>>       
>
  ---------------------------------------------------------------------
>   
>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>> For additional commands, e-mail:
 users-help at gridengine.sunsource.net
>>>
>>>   
>>>     
>>>       
>>
 ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>>
>>
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam?  Yahoo! Mail has the best spam protection around 
>> http://mail.yahoo.com 
>>
>>
 ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>   
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net





__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list