[GE users] Defining priority of hosts

Kirk Patton kpatton at transmeta.com
Wed Aug 30 14:35:18 BST 2006


I used load based sorting to get jobs dispatched to my fast hosts first.
Each host has a complex named 'relative_speed', and my 'load_formula'
uses that number.

Ex.
queue_sort_method                 load
load_formula                      np_load_avg-relative_speed

The 'relative_speed' is exactly that.  A finger-to-the-wind number
that gives a very rough comparison of the processing power of one host
compared to another.

To keep things consistent, I wrote a little perl script to generate the 
relative_speed number.  It generates some random data and times how quickly 
it can compress it.  That number becomes the 'relative_speed' number.

The script should be run several times, when a host is idle, to get a accurate measurement.

I have included the script is case it is of use to anyone.

Kirk

################################################################################################3
#!/usr/bin/perl

use Time::HiRes qw( gettimeofday );
use strict;

my $debug = 0;
my $out_directory = "/tmp";
my $block_sz      = 1024;
my $outfile       = "$out_directory/random.dat";
my $bzip2         = "/usr/bin/bzip2";
my $count         = 0;

print relative_speed(),"\n";

sub relative_speed{
#Calculate the speed of the host to be use for comparisons

    #Create the file to compress
    open(FILE, ">$outfile") or die "Can't open $outfile for write: $!\n";
    while( $count <= 500 ){
        my $blk = &block($block_sz);
        syswrite FILE,$blk or die "Can't write to $outfile: $!\n";
        ++$count;
    }
    close FILE;

#Get start time
   my($start_sec,$start_micro) = gettimeofday;

#compress the file
    if( system( "$bzip2 -f -9 $outfile" ) == -1 ){ die "Could not compress file: $!\n";}

#Get stop time
    my($end_sec,$end_micro) = gettimeofday;

#Print difference in time
    my $start = "$start_sec.$start_micro";
    my $end   = "$end_sec.$end_micro";
    my $dif = $end - $start;
    $dif = (1/$dif) * 10;
    $dif = sprintf("%00d", $dif);

    if( $debug ){
        print "Compressed file in $dif second(s)\n";
        print "relative_speed = $dif\n";
    }

    unlink("$outfile.bz2");
    return $dif;



}

sub block{
        #Return a random string that is $block in length
    my $block_sz = shift;
    my($byte,$block);
    my $count = 0;
    while($count <= $block_sz){
        ++$count;
        $byte = int(rand(1)*10);
        if($byte >= 5){ $byte = 1; }
        else{ $byte = 0; }
        $block .= $byte;

    }
    return $block;
}

END {
    unlink $outfile;
    unlink "${outfile}.bz2";
}
################################################################################################3


On Tue, Aug 29, 2006 at 07:02:43PM -0400, Chris Dagdigian wrote:
> 
> One method to do  this would be to switch from load based sorting to  
> one based on sequence numbers. You can then number hosts with a  
> sequence number that reflects relative speed.
> 
> This may be a good start:
> http://blogs.sun.com/sgrell/entry/n1ge_6_scheduler_hacks_sorting
> 
> -Chris
> 
> 
> 
> On Aug 29, 2006, at 5:09 PM, Bisbal, Prentice wrote:
> 
> >Greetings, everyone.
> >
> >I'm a GridEngine newbie, so please forgive me if this is a simple/ 
> >dumb question. I searched through the docs, and couldn't find an  
> >answer there. I could have been using the wrong terminology in my  
> >searches.
> >
> >I'm not sure if "priority" is the correct term in GridEngine  
> >parlance. Here's what I'm trying to do: My execution hosts are a  
> >mixture of different systems, from old SGIs to new multi-core,  
> >multi-processor Opterons. All systems are in the same queue. When  
> >jobs are submitted, I'd like them to be assigned to the fasted idle  
> >computer available, instead of randomly assigned or round-robbined.
> >
> >For example, if host A has 4 processors. Host B has 8 processors,  
> >but those processors are only 1/2 as fast as host A. If I submit 5  
> >jobs, I want all 4 processors of  host A filled before jobs are  
> >assigned to host B.
> >
> >What is the best way to do this?
> >
> >
> >Prentice
> >
> >
> >
> >The contents of this communication, including any attachments, may  
> >be confidential, privileged or otherwise protected from disclosure.  
> >They are intended solely for the use of the individual or entity to  
> >whom they are addressed. If you are not the intended recipient,  
> >please do not read, copy, use or disclose the contents of this  
> >communication. Please notify the sender immediately and delete the  
> >communication in its entirety.
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

-- 
Kirk Patton
Unix Administrator
Transmeta Inc.
Tel. 408 919-3055

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list