[GE users] Defining priority of hosts

Bisbal, Prentice PBisbal at LexPharma.com
Wed Aug 30 19:12:49 BST 2006


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

I notices that in the "Execution Hosts" tab of the Host Configuration window, there is section called "Load Scaling" what does this do? 

Prentice 


-----Original Message-----
From: Kirk Patton [mailto:kpatton at transmeta.com]
Sent: Wed 8/30/2006 9:35 AM
To: users at gridengine.sunsource.net
Subject: Re: [GE users] Defining priority of hosts
 
I used load based sorting to get jobs dispatched to my fast hosts first.
Each host has a complex named 'relative_speed', and my 'load_formula'
uses that number.

Ex.
queue_sort_method                 load
load_formula                      np_load_avg-relative_speed

The 'relative_speed' is exactly that.  A finger-to-the-wind number
that gives a very rough comparison of the processing power of one host
compared to another.

To keep things consistent, I wrote a little perl script to generate the 
relative_speed number.  It generates some random data and times how quickly 
it can compress it.  That number becomes the 'relative_speed' number.

The script should be run several times, when a host is idle, to get a accurate measurement.

I have included the script is case it is of use to anyone.

Kirk

################################################################################################3
#!/usr/bin/perl

use Time::HiRes qw( gettimeofday );
use strict;

my $debug = 0;
my $out_directory = "/tmp";
my $block_sz      = 1024;
my $outfile       = "$out_directory/random.dat";
my $bzip2         = "/usr/bin/bzip2";
my $count         = 0;

print relative_speed(),"\n";

sub relative_speed{
#Calculate the speed of the host to be use for comparisons

    #Create the file to compress
    open(FILE, ">$outfile") or die "Can't open $outfile for write: $!\n";
    while( $count <= 500 ){
        my $blk = &block($block_sz);
        syswrite FILE,$blk or die "Can't write to $outfile: $!\n";
        ++$count;
    }
    close FILE;

#Get start time
   my($start_sec,$start_micro) = gettimeofday;

#compress the file
    if( system( "$bzip2 -f -9 $outfile" ) == -1 ){ die "Could not compress file: $!\n";}

#Get stop time
    my($end_sec,$end_micro) = gettimeofday;

#Print difference in time
    my $start = "$start_sec.$start_micro";
    my $end   = "$end_sec.$end_micro";
    my $dif = $end - $start;
    $dif = (1/$dif) * 10;
    $dif = sprintf("%00d", $dif);

    if( $debug ){
        print "Compressed file in $dif second(s)\n";
        print "relative_speed = $dif\n";
    }

    unlink("$outfile.bz2");
    return $dif;



}

sub block{
        #Return a random string that is $block in length
    my $block_sz = shift;
    my($byte,$block);
    my $count = 0;
    while($count <= $block_sz){
        ++$count;
        $byte = int(rand(1)*10);
        if($byte >= 5){ $byte = 1; }
        else{ $byte = 0; }
        $block .= $byte;

    }
    return $block;
}

END {
    unlink $outfile;
    unlink "${outfile}.bz2";
}
################################################################################################3


On Tue, Aug 29, 2006 at 07:02:43PM -0400, Chris Dagdigian wrote:
> 
> One method to do  this would be to switch from load based sorting to  
> one based on sequence numbers. You can then number hosts with a  
> sequence number that reflects relative speed.
> 
> This may be a good start:
> http://blogs.sun.com/sgrell/entry/n1ge_6_scheduler_hacks_sorting
> 
> -Chris
> 
> 
> 
> On Aug 29, 2006, at 5:09 PM, Bisbal, Prentice wrote:
> 
> >Greetings, everyone.
> >
> >I'm a GridEngine newbie, so please forgive me if this is a simple/ 
> >dumb question. I searched through the docs, and couldn't find an  
> >answer there. I could have been using the wrong terminology in my  
> >searches.
> >
> >I'm not sure if "priority" is the correct term in GridEngine  
> >parlance. Here's what I'm trying to do: My execution hosts are a  
> >mixture of different systems, from old SGIs to new multi-core,  
> >multi-processor Opterons. All systems are in the same queue. When  
> >jobs are submitted, I'd like them to be assigned to the fasted idle  
> >computer available, instead of randomly assigned or round-robbined.
> >
> >For example, if host A has 4 processors. Host B has 8 processors,  
> >but those processors are only 1/2 as fast as host A. If I submit 5  
> >jobs, I want all 4 processors of  host A filled before jobs are  
> >assigned to host B.
> >
> >What is the best way to do this?
> >
> >
> >Prentice
> >
> >
> >
> >The contents of this communication, including any attachments, may  
> >be confidential, privileged or otherwise protected from disclosure.  
> >They are intended solely for the use of the individual or entity to  
> >whom they are addressed. If you are not the intended recipient,  
> >please do not read, copy, use or disclose the contents of this  
> >communication. Please notify the sender immediately and delete the  
> >communication in its entirety.
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

-- 
Kirk Patton
Unix Administrator
Transmeta Inc.
Tel. 408 919-3055

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net






The contents of this communication, including any attachments, may be confidential, privileged or otherwise protected from disclosure.  They are intended solely for the use of the individual or entity to whom they are addressed.  If you are not the intended recipient, please do not read, copy, use or disclose the contents of this communication.  Please notify the sender immediately and delete the communication in its entirety.



    [ Part 2: "Attached Text" ]

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net



More information about the gridengine-users mailing list