[GE dev] Another qacct performance improvement

roland roland.dittel at sun.com
Wed May 13 15:04:08 BST 2009


Rayson,

thanks for the additional enhancements. I've made some small changes and 
   put the code into the 6.2u3 source repository.

Roland

rayson wrote:
> A much smaller performance improvement than Roland's fix for issue
> 2510. The changes are:
> 
> 1) In sge_read_rusage(), the static buffer szLine[] is removed.
> Instead, the caller is responsible for allocating a buffer and passing
> it to sge_read_rusage(). While it is not a big issue for now, this
> (plus if we use strtok_r()) will enable multiple threads to parse the
> accounting file in the future.
> 
> 2) Further defer allocating and copying the fields (the sge_strdup()
> block at the end of
> sge_read_rusage()). Currently, when a line is of interest,
> sge_read_rusage() calls sge_strdup() to duplicate the fields. And then
> the strings are freed in the next iteration. This means parsing the
> accounting file can cause a lot of allocate and free. The new way of
> doing is to keep all the fields in the buffer passed in from the
> caller.
> 
> I use the extreme case, which is when every line matches the condition
> set by the user, the performance can improve by 30%:
> 
> Latest CVS:
> % /usr/bin/time qacct
> Total System Usage
>     WALLCLOCK         UTIME         STIME           CPU
> MEMORY                 IO                IOW
> ================================================================================================================
>        215775     86310.000     21577.500      9000.000
> 9000.000              0.000              0.000
> 
> 0.82user 0.00system 0:01.55elapsed 53%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+622minor)pagefaults 0swaps
> 
> With my change:
> % /usr/bin/time qacct
> Total System Usage
>     WALLCLOCK         UTIME         STIME           CPU
> MEMORY                 IO                IOW
> ================================================================================================================
>        215775     86310.000     21577.500      9000.000
> 9000.000              0.000              0.000
> 0.64user 0.00system 0:01.16elapsed 56%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+623minor)pagefaults 0swaps
> 
> I've uploaded my version of qacct.c in issue 2510.
> 
> Rayson
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=39&dsMessageId=194862
> 
> To unsubscribe from this discussion, e-mail: [dev-unsubscribe at gridengine.sunsource.net].


-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Roland Dittel               Tel: +49 (0)941 3075-275 (x60275)
Software Engineering        Fax: +49 (0)941 3075-222 (x60222)
Sun Microsystems GmbH
Dr.-Leo-Ritter-Str. 7       mailto:roland.dittel at sun.com
D-93049 Regensburg          http://www.sun.com/gridware
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Registered Office / Sitz der Gesellschaft:
   Sun Microsystems GmbH
   Sonnenallee 1
   D-85551 Kirchheim-Heimstetten
   Germany
Commercial register of the Local Court of Munich /
Handelsregistereintrag Amtsgericht Muenchen:
   HRB 161028
Managing Directors / Geschaeftsfuehrer:
   Thomas Schroeder, Wolfgang Engels, Wolf Frenkel
Chairman of the Supervisory Board / Vorsitzender des Aufsichtsrates
   Martin Haering

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=39&dsMessageId=195041

To unsubscribe from this discussion, e-mail: [dev-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list