[GE users] xml-qstat and FlexLM

Chris Dagdigian dag at sonsorol.org
Tue May 13 15:06:50 BST 2008


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]


Ok. Mark's kind words have convinced me to post here (one more time at  
least!)

I'm currently in Oakland, CA attending the Grid Engine workshop track  
at this conference:

http://www.opensourcegridcluster.org/

Lots of cool people are here including Roland, Fritz, Miha and DanT --  
where is everyone else :)

I'm scheduled to give my "Fun with Grid Engine XML" talk later on this  
afternoon. That has given me a good reason to get back into the xml- 
qstat codbase and finally see all the improvements and cleanup that  
Mark has provided. It's my fault for essentially halting work on this  
code as the SGE Issue #2335 (load_avg data missing from "qstat -f - 
xml") has lingered untouched and unfixed for ~10 months now. I'm much  
happier now with the recent discussion on this list about altering the  
XML output to make it more usable and excited that Michael Pospisil  
has been actively soliciting feedback. I think we finally have a  
critical mass of end-users who want to work with SGE XML output and we  
are now in a position to provide helpful and productive feedback and  
tips to the developers.

Anyway, with SGE 6.2beta coming out this week and the SGE folks  
presenting the roadmap and other big picture info at the conference  
this week it's actually a good time to re-dedicate some effort to  
cleaning up xml-qstat and kicking a 1.0 release out the door.

I'm going to ask for help at my talk in sorting out the Cocoon caching  
issue. I also think I should commit to a 1.0 release at the same time  
that SGE 6.2 goes out the door in final form (August'08 I think?). By  
that time we'll either have the Cocoon generator issues solved or  
we'll have been committed to a daemon-based  XML caching process.

That also gives me time to fool around with the iPhone SDK to see if I  
can wedge an xml-qstat interface into the iPhone which could be a fun  
little diversion.

If anyone on this list has been using xml-qstat and feels like  
contributing patches or fixes just drop me a line. Right now there are  
only 3 people with SVN commit access and it would be nice to grow that  
list a bit.

-Chris







On May 13, 2008, at 1:25 AM, Olesen, Mark wrote:

> Hi Christopher,
>
> Don't worry, the FlexLM stuff is definitely not required for xml- 
> qstat,
> but my slant probably does tend to show a bit in there restructured
> code. Based on your note though, I've finally made a small change to
> allow you to hide the qlicserver resources icon (the folder with the
> database picture) based on the value in the CONFIG.xml file.  I'll get
> it checked in as soon as the machine that I've been using for my ssh
> tunnel gets working.
>
> Just to add to Chris' comments:
>
>> What you need to do is alter the redirect line so that it forces the
>> browser to "qstat-jobs.html" which is the rendered view that does not
>> depend on cached XML data from the qlicserver code.
>
> There are 3 different means of generating the xml source data:
> 1) "qstat -xml" output that has been cached from the qlicserver.
> 2) "qstat -f -xml" output generated on-the-fly with the Java
> CommandGenerator.
> 3) "qstat -f -xml" output generated via a daemon and cached to a file.
>
> The entry point "jobs.html" corresponds to #1 and the entry point
> "qstat-jobs.html" corresponds to #2 or #3. Thus, as Chris mentioned,
> you'll have to decide which entry point you'd like to get redirected  
> to
> for a plain "xmlqstat/" url.
>
>> (2) The 2nd change you'll need to do in sitemap is a global search  
>> and
>> replace on "qstat.sh" and replace it with just "qstat". We have
>> qstat.sh in the sitemap file as a debug step so that we can log
>> whenever Cocoon makes a Grid Engine call. This was originally used to
>> confirm by Mark that Cocoon  was not actually smartly caching XML and
>> was directly calling qstat each time the page was reloaded (something
>> we want to avoid).
>
> Based on these results I would really recommend the continued use of  
> the
> older caching mechanism for now. The corresponding daamon and start
> scripts can be found under "scripts/sge-xml-cacher.pl" and
> "init-scripts/sge-xml-cacher", respectively.
>
> Since the CommandGenerator caching is not working as it should. Every
> load/reload request from each browser will trigger a complete "qstat  
> -f
> -xml" request. Even if this doesn't bother your qmaster much, I still
> managed to annoy cocoon and/or java enough to receive a "too many  
> files
> open" error that request a complete shutdown/restart of cocoon/java.  
> The
> "qstat -j ..." request is probably okay to leave as is, since it won't
> be hit quite as often.
>
> Once the daemon is running, you can change from the on-demand  
> mechanism
> to the cached mechanism as mentioned in the sitemap.xmap comments:
>
>  <!--
>      The full qstat query traditionally used by xmlqstat and generated
>      by the sge-xml-cacher.pl daemon.
>      Use a symbolic link to point to the file generated by the daemon.
>
>      If you are using the cached information instead of the
>      CommandGenerator,
>      search and replace qstatfCmd with qstatfFile below.
>  -->
>
>
> I've left the discussion here on the GridEngine list since there will
> undoubted more people with very similar issues.  Besides which, I  
> think
> that the xml-qstat interface is much more useful for the end-user that
> Chris' modesty allows him to admit.
>
> /mark
> This e-mail message and any attachments may contain
> legally privileged, confidential or proprietary Information,
> or information otherwise protected by law of EMCON
> Technologies, its affiliates, or third parties. This notice
> serves as marking of its ?Confidential? status as defined
> in any confidentiality agreements concerning the sender
> and recipient. If you are not the intended recipient(s),
> or the employee or agent responsible for delivery of this
> message to the intended recipient(s), you are hereby
> notified that any dissemination, distribution or copying
> of this e-mail message is strictly prohibited.
> If you have received this message in error, please
> immediately notify the sender and delete this e-mail
> message from your computer.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list