[GE users] CPU usage by array jobs

Pascal Wassam pascal at blur.compbio.ucsf.edu
Thu Jun 21 20:59:31 BST 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

I would like to second all the experiences Iwona has written about here. 
I will also attempt to conduct some tests and present something that is 
repeatable for developers to play with.

-Pascal

Iwona Sakrejda wrote:
> Since this is a somehow different problem I gave it a new title.
>
> Rayson Ho wrote:
>>> Another problem I am having is that array jobs seem to be overcharged
>>> when the usage is calculated (could you point me to the section of 
>>> code that
>>> deals with it/ I'll be happy to read it). Looks like each array job 
>>> gets
>>> the CPU usage of the whole array. Array jobs are very helpful but 
>>> users are
>>> fleeing from them in droves.....
>>
>> How to reproduce it?? Is it a parallel or serial job??
> It happens to serial jobs. I have not done thorough studies yet, but I 
> see that
> usage for owners of array jobs greatly exceeds what I estimate it 
> should be.
>
> Also when I clear usage, then only the usage from that moment should be
> taken into account - right? And I see that a user who has an array 
> jobs, gets
> right away usage that exceeds what he has running at that point.
>
> Another shred of evidence is that when they switch from array jobs to
> individual jobs, they get a throughput that they feel is consistent 
> with their share.
> If they use arrays their throughput dives.
>
> I'll try to come with a clean example with numbers.  It is in 6.0u4 so 
> since
> I have to upgrade anyway I was postponing more studies hoping that the 
> upgrade will
> fix the problem. On the other hand it might not and it really 
> increases  the load
> when instead of 1 array job with  1000 members I get 1000 jobs.....
>
> And today I noticed that discussion about shares and CPU consumption so
> I hoped the right expert might be watching and it would be easy for 
> him to look at it...
>
> Iwona
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list