[GE users] Accounting

Shaila Parashar shaila at engr.colostate.edu
Thu Apr 21 17:39:41 BST 2005


I have installed ganglia and SRS and am doing a qhost/qstat regularly 
now. This will give me statistics for jobs that are sent after I 
installed it. I also would like to get the statistics for jobs that I 
sent in the last 6 months and so I guess I will have to do as you 
suggested - doing some data mining on the accounting file.  
I will also look at ARCo.

Will inform you all of what turned out to be the best option for us.

Thanks for your response.

Shaila


Ron Chen wrote:

>Richard,
>
>(Got your mail while writing a reply to Shaila's
>original message)
>
>Does ARCo report why the job isn't scheduled
>immediately (answers can be: large number of queued
>jobs ahead of the job, and/or resource requirements
>not satisfied. May be giving the users the % of wait
>time contributed by each of the reasons is useful!)
>
>Anyway, here's my original mail:
>
>I guess you can get the number of CPUs used by looking
>at the accounting file, and find out when the jobs are
>running, and how many CPUs are used by each one. This
>requires some sort of data mining.
>
>I think it may be easier to run qstat/qhost
>periodically and store the output to a file/database.
>Doing this would then allow us to find out the license
>usage (or any kind of resources that change
>dynamically), and why the job isn't dispatched
>(because of many jobs ahead of it, or because the job
>requests for a resource that can't be obtained).
>
>BTW, did you look at ARCo?
>
> -Ron
>
>
>--- Richard Hierlmeier wrote:
>  
>
>>Shaila,
>>
>>the commercial version of the gridengine includes
>>the Accounting and Reporting
>>Console. It also includes a daemon which imports the
>>reporting and accounting data
>>into a realation database (postgresql an oracle is
>>supported).
>>This tools uses the reporting file
>>($SGE_ROOT/$SGE_CELL/common/reporting) not the
>>accounting file. All information you need should be
>>included in this file.
>>
>>Regards
>>
>>   Richard
>>
>>Shaila Parashar wrote:
>>    
>>
>>>Hi
>>>
>>>We have a cluster consisting of 4 nodes  running
>>>      
>>>
>>SGE 6.0u3 .  The number 
>>    
>>
>>>of CPUs on these nodes are 24, 12, 8 and 4 . We
>>>      
>>>
>>have been running this 
>>    
>>
>>>cluster since Aug 2004 and are now in need of some
>>>      
>>>
>>statistics. I read 
>>    
>>
>>>the mailing lists and did get some ideas about the
>>>      
>>>
>>statistics. I also 
>>    
>>
>>>imported the accounting file into MySQL . I did
>>>      
>>>
>>manage to get statistics 
>>    
>>
>>>on the jobs - as average CPU time used, average
>>>      
>>>
>>waiting time, etc. But 
>>    
>>
>>>we need statistics similar to the following :-
>>>
>>>Number of CPU's used versus time. Also number oc
>>>      
>>>
>>CPUs used on each of 
>>    
>>
>>>the hosts vs time.
>>>
>>>Basically we need plots vs time.
>>>
>>>I wanted to know if this is possible from the SGE
>>>      
>>>
>>accounting files and 
>>    
>>
>>>if so how ?
>>>
>>>I would appreciate it if  you can suggest of any
>>>      
>>>
>>other values that we 
>>    
>>
>>>can plot against time.
>>>
>>>Any ideas/suggestions on how to get these values (
>>>      
>>>
>>if possible ) will be 
>>    
>>
>>>really appreciated.
>>>
>>>Thanks
>>>
>>>Shaila
>>>
>>>      
>>>
>>-- 
>>- - - - - - - - - - - - - - - - - - - - - - - - - -
>>- - - - - - - - - - - -
>>Richard Hierlmeier           Phone: ++49 (0)941
>>3075-223
>>Software Engineering         Fax:   ++49 (0)941
>>3075-222
>>Sun Microsystems GmbH
>>Dr.-Leo-Ritter-Str. 7	     mailto:
>>richard.hierlmeier at sun.com
>>D-93049 Regensburg           http://www.sun.com/grid
>>
>>
>>    
>>
>---------------------------------------------------------------------
>  
>
>>To unsubscribe, e-mail:
>>users-unsubscribe at gridengine.sunsource.net
>>For additional commands, e-mail:
>>users-help at gridengine.sunsource.net
>>
>>
>>    
>>
>
>
>		
>__________________________________ 
>Do you Yahoo!? 
>Yahoo! Mail - Find what you need with new enhanced search. 
>http://info.mail.yahoo.com/mail_250
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net
>  
>

-- 
*****************************************************************
Shaila Parashar			e-mail:shaila at engr.colostate.edu
UNIX System Administrator	tel:- (970)-491-6555
Engineering Network Services
Colorado State University
Fort Collins, CO 80523-1301
******************************************************************
" Smile is a curve that sets things straight. " 




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list