[GE users] qsub preventing -clear (JSV)

fredwag Frederik.Wagner at lrz.de
Wed Jul 14 16:27:18 BST 2010


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/14/2010 03:59 PM, templedf wrote:
> This should explain the performance issue:
> 
> http://blogs.sun.com/templedf/entry/performance_considerations_for_jsv_scripts

thanks! sounds like server side JSV and perl is a good option!

> 
> Daniel
> 
> On 07/14/10 06:49 AM, reuti wrote:
>> Hi Frederik,
>> 
>> Am 14.07.2010 um 15:31 schrieb fredwag:
>> 
>>> On 07/14/2010 03:21 PM, reuti wrote:
>>>> Hi,
>>>> 
>>>> Am 14.07.2010 um 14:51 schrieb fredwag:
>>>> 
>>>>> in the SGE documentation it is stated, that server side JSVs 
>>>>> shouldn't be scripts. But what are the alternatives? Should
>>>>> one write a C program or is perl already o.k. (so script
>>>>> meaning bash or similar?).
>>>> 
>>>> it's just a performance issue. Nevertheless you can use scripts
>>>> - how many jobs do you expect to be submitted per second? When
>>>> you start
>>> 
>>> not too many, it's more some like some thousands over the day
>>> (which _could_ come in some bulks)
>> 
>> then I woud suggest to start with a script and observe the load on
>> the qmaster machine. It might also depend on the usual runtime of a
>> job. For us jobs run usually for days, hence whether the submission
>> takes a few seconds more or less is not an issue.

it's similar here, it can just happen that somebody launches a pile of
serial jobs from a script: I just want to be sure the qmaster does not
get overloaded by this.

Bye,
Frederik

>> 
>> 
>>>> from scratch with e.g. a pure C program, you would need to
>>>> follow the exact communication protocol for a JSV, while this
>>>> burden is taken from you with the included libs for bash, perl,
>>>> tcl and JAVA.
>>>> 
>>>> When you implement the communication protocol, you can use any 
>>>> programming language you like.
>>> 
>>> I was expecting this. I was skeptic due to the docs stating only
>>> the support of 'script:/...'
>> 
>> You mean the Wiki? In the Wiki the communication protocol is listed
>> in detail.
>> 
>> -- Reuti
>> 
>> 
>>>>> So not having answered the previous question, for a more
>>>>> complex JSV it look like using client side JSV is the right
>>>>> thing. When I do some mandatory mangling with the qsub
>>>>> option: How do I prevent a user from using the '-clear' flag
>>>>> in which case any administrator defined JSV (in
>>>>> $SGE_ROOT/$SGE_CELL/common/sge_request) will not be
>>>>> executed?
>>>> 
>>>> Although the JSV shouldn't fork, it's still possible to use a
>>>> bash wrapper which start a more complex computation in C when
>>>> it's bulky.
>>> 
>>> it's not so bulky. I got a bit skeptic due to the docs mentioning
>>> the performance issue. So you would say a server side JSV is
>>> normally fine (I would use perl then), since there's no way to
>>> prevent the '-clear'?
>>> 
>>> Thanks a lot, Frederik
>>> 
>>>> 
>>>> -- Reuti
>>>> 
>>>> 
>>>>> To me it looks like: either server side JSV with performance 
>>>>> problems or client side JSV with the possible user
>>>>> interference...
>>>>> 
>>>>> Thanks a lot for any help, Frederik
>>>>> 
>>>>> ------------------------------------------------------ 
>>>>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=267965
>>>>>
>>>>>
>>>>>
>>>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkw911YACgkQ7zXn0gvOeag78QCg7e15LmRgPBg9xeiQ9kNVtHNf
vkQAn3j88SLQ4ut5k+JFJJLCkaPLmO58
=cGuN
-----END PGP SIGNATURE-----

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=267993

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list