[GE users] move qmaster's spool location ? & performance

cjf001 john.foley at motorola.com
Fri Apr 23 20:04:53 BST 2010


OK, to followup on this....

the move of the qmaster's spool area off of the NFS share
worked fine - but it didn't solve my performance issue.

(performance issue = ~8500 jobs in system, all "q" commands run
verrrry slow - ie, ~30-40 seconds for a qstat. When the number of
jobs in the system drops to ~6900, qstat returns in ~5 - 10
seconds - much better.)

So, I'm beginning to think that I'm going to have to switch to
the Berkeley db backend, instead of using flat files, if the
users are going to be dumping so many jobs into the system.


So, a few questions for you all !

- I'm running all SGEv6.2u2 right now. Can I upgrade just the qmaster
   to v6.2u5, and leave the clients at 6.2u2 ?  If so, can the clients
   be a mix of 6.2u2 and 6.2u5 ? (would make the eventual upgrade
   easier....)

- Is there a way to limit the number of jobs in the system - that is,
   keep qsub from submitting if the total number of jobs is greater
   that some number ? I know there's a limit on *running* jobs, but
   I couldn't find anything on *running + pending* jobs.

- anybody else out there running the flat file qmaster backend, with
   > 5000 jobs in the system (running + pending) ?  If so, how's
   your performance ?


     Thanks,

        John


John Foley wrote:
> I guess I could try that.... my simlink would actually be on the
> NFS server, though, so I'm not sure if that would eliminate the
> NFS traffic for my test....
>
> I found that the spooler location is defined in the "bootstrap"
> file... I'm thinking of just stopping the qmaster, changing the
> entry in the bootstrap file, copying the data to the new local
> location, and then restarting qmaster.....
>
> Anyone see anything wrong with that ?
>
> Thanks,
>
> John
>
> rayson wrote:
>> On Thu, Apr 22, 2010 at 4:14 PM, cjf001 wrote:
>>> I ask because my current qmaster spool directory is on our NFS server,
>>> but I'm thinking I could (temporarily, at least) move it to the
>>> local machine's disk to see if that's causing my performance issue.
>>
>> A quick and dirty way: with a sym. link and you can place your spool
>> directory wherever you want.
>>
>> Rayson
>>
>>
>>
>>
>>>
>>> Thanks,
>>>
>>> John
>>>
>>>
>>>
>>> --
>>> ###########################################################################
>>>
>>> # John Foley # Location: IL93-E1-21S #
>>> # IT& Systems Administration # Maildrop: IL93-E1-35O #
>>> # Antenna& Mechanical Simulation Grp # Email: john.foley at motorola.com #
>>> # Motorola, Inc. - Mobile Devices # Phone: (847) 523-8719 #
>>> # 600 North US Highway 45 # Fax: (847) 523-5767 #
>>> # Libertyville, IL. 60048 (USA) # Cell: (847) 460-8719 #
>>> ###########################################################################
>>>
>>> (this email sent using SeaMonkey on Windows)
>>>
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=254501
>>>
>>>
>>> To unsubscribe from this discussion, e-mail:
>>> [users-unsubscribe at gridengine.sunsource.net].
>>>
>>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=254503
>>
>>
>> To unsubscribe from this discussion, e-mail:
>> [users-unsubscribe at gridengine.sunsource.net].
>
>
>



-- 
###########################################################################
# John Foley                          # Location:  IL93-E1-21S            #
# IT & Systems Administration         # Maildrop:  IL93-E1-35O            #
# Antenna & Mechanical Simulation Grp #    Email: john.foley at motorola.com #
# Motorola, Inc. -  Mobile Devices    #    Phone: (847) 523-8719          #
# 600 North US Highway 45             #      Fax: (847) 523-5767          #
# Libertyville, IL. 60048  (USA)      #     Cell: (847) 460-8719          #
###########################################################################
               (this email sent using SeaMonkey on Windows)

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=254658

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list