[GE users] how to find available slots for multiple pe assigned to a single queue

johnny laynejg at vcu.edu
Tue Oct 10 20:28:44 BST 2006


hi,
I think a PE is just sort of a set of rules for an environment for doing 
parallel work. Perhaps what you mean is you'd like to know how many 
slots are available in queues referencing certain PEs?
I'm actually fiddling with some Perl code (but you can use C/C++ & 
popen, etc.) at the moment that does just these sorts of things. To see 
how many slots are available to a queue:

1. This actually returns a list of {slots, load} per queue instance, but 
will give you the idea; you can then take this list and sort it or 
whatever you like:
# get the name of the queue to investigate
my $q = $_[0];
# prepare an empty array to store the info
my @slots_and_loads = ();
my $test = 1;

# open a qstat process to search for info on the queue
open(QSTAT, "qstat -f | grep $q |") or $test = 0;
if (0 == $test) {
print "Failed to open qstat process! Couldn't determine the # slots open 
in $q...\n";
# be sure to check for this returned error value!
return ();
}
while (<QSTAT>) {
chomp($_);
my @wrds = split(" ", $_);
# make sure queue instance not in an unruly state
if (5 == @wrds) {
# prepare queue info
my $slots = (split("/", $wrds[2]))[0];
# store the number of open slots for this queue instance
push @slots_and_loads, "$slots";
# store the load on this queue instance --> NOTE: for some purposes, it 
might be better to use rsh to get this....
push @slots_and_loads, $wrds[3]; # you could also push the name of the 
queue instance here if you like
}
}
close(QSTAT);
return @slots_and_loads;

With this list or something similar in hand, you can sum the number of 
available slots, determine the current load, and so on to make your 
decisions about what queues/queue instances to use.

2. Similarly, to get a list of PEs available to a queue:
open(QCONF, "qconf -sq $q |") or return ();
while (<QCONF>) {
.... Search for a line with ""pe_list", pick out any PEs listed; I 
suggest running qconf -sq $q at command line to get a feel for this ....
.... push any PEs found onto a list ...
}

You can use similar code to see what PEs are available per user, as 
well. Just study the qconf output.

3. Now you can get info for each PE in the list, to see how many slots 
are available for each PE & so on:
open(QCONF, "qconf -sp $pe |") or $test = 0;
if (0 == $test) {
print "in get_num_slots(): Couldn't open \"qconf -sp $pe\"" .
"process...\n";
# be sure to check for this error code!
return -1;
}
else {
while (<QCONF>) {
chomp($_);
if ($_ =~ "slots") {
$slots = (split " ", $_)[1];
last;
}
}
close(QCONF);
}
return $slots;
but this doesn't tell you how many slots are being used, just what's 
available to your PE setup.

I hope that can help give you enough info to do what you want. You can 
use (1.) to see if your queue has enough available slots, (2.) to make 
sure there's a PE defined that the queue can use, and (3.) to make sure 
the PEs for the queue suit your needs. I hope that helps! I'm pretty 
much a newb here myself so hopefully someone has much easier answers for 
you, but code like this isn't really that difficult. Good luck, and keep 
us posted.
johnny


SLIM H.A. wrote:
> We have an ethernet and a myrinet host group, each with their special
> pe. 
> I created a single queue from these two groups, as the user has to
> specify the pe anyway when requesting the nslots with the -pe option.
> However qmon displays all used/available slots in the queue and does not
> separate between pe. 
> I cannot find a simple way to discover the available slots for each pe
> in the queue (eg with qstat)  but maybe someone has done this before?
>
> A solution would be to create a separate queue for each of the pe's/host
> groups but that seems 
> an unnecessary proliferation of queues?
>
> Thanks
>
> Henk
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list