[GE users] web interface to "qstat -xml" -- any java folk able to lend a hand? (need a cocoon generator for qstat)

Joe Landman landman at scalableinformatics.com
Fri Nov 11 02:02:56 GMT 2005

    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]


[use the right Perl modules Luke...  XML::Smart in this case ]

   Our 6.0u6 perl based parser fits into a single line, after we grab 
the data.

	$qstat=`/opt/gridengine/bin/lx24-amd64/qstat -xml`;
	$xml    = XML::Smart->new($qstat);

(no schema/DTD needed)

then for example, iterating over all the jobs  ...

	foreach ($xml->{job_info}->{queue_info}->{job_list}('@') )

We are doing this with HTML::Mason (http://www.masonhq.com) so we have 
Perl interspersed with the HTML.  If you would like to see this code, 
please fire me a note.   We are using this as a way to generate a table 
of all jobs in flight, in queue, and calculate for every job in flight 
how much time it has spent computing.  With a little extra effort, we 
can get process memory and other utilization, but our customers haven't 
asked us for this.

So far it is working nicely on a system with 144 cores and well over 800 
jobs queued and 140 in flight, with a 5 second refresh.   This system 
showed the need for pagination which I haven't done (yet), but ...

If you need to use the Java bits due to your coding requirements, you 
would need to use the other bits as indicated.  I am sure they work well.


Chris Dagdigian wrote:
> I've junked most of the perl CGIs and scripts responsible for  
> collecting and transforming SGE XML and have started re-developing  the 
> little "xml-qstat" interface under the Apache Cocoon XML  publishing 
> framework.
> A work in progress demo is here:
> http://dcore-amd.sonsorol.net:8888/xmlqstat/qstat.html
> The output format above is biased towards the "show me everything"  side 
> which works for small clusters. XML is dynamically collected  from SGE 
> at periodic intervals via an external caching daemon.
> For very large and active clusters where the "show me everything"  model 
> breaks down ( or breaks browsers! ) I've started work on some  "terse" 
> XSL stylesheets. The effect of the "terse" stylesheet as  applied to 
> canned XML taken from a largish cluster can be seen here:
> http://dcore-amd.sonsorol.net:8888/xmlqstat/demo/qstat-terse.html
> The reason for this email is that I need to start work on the more  
> dynamic stuff -- for instance the ability to click on a SGE JobID and  
> have rendered output from a dynamic "qstat -j <jobID>" call appear  
> within a DHTML DIV element.
> The "proper" way to do this within Apache Cocoon would be to write a  
> java "Cocoon Generator" capable of calling out to  "qstat -xml" and  
> bringing the resulting XML directly into a Cocoon XML object for  
> transformation and XHTML serialization.
> But ...
> I'm a perl geek, not a java software engineer. I have absolutely no  
> idea how to go about writing a qstat XML generator for Apache Cocoon.  
> I'm pretty sure its not all that hard to do but I'm just not capable  of 
> getting my head around the problem.
> If I can't figure out the pure-java-Cocoon methods I'm just going to  
> write some tiny perl CGI's and have Cocoon fetch the dynamic XML via  
> http operations which is not as efficient (or elegant!) The perl  stuff 
> I can spin out easily but I'd rather this be a 100% Cocoon  package if I 
> can manage it.
> Anyone willing to help me out with this? The xml-qstat stuff is open  
> source under a creative commons license. The Cocoon Generator  approach 
> will make the resulting package far easier to setup and  distribute 
> since external CGIs for XML data fetching won't be needed.
> -Chris
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452
cell : +1 734 612 4615

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list