[GE users] submitting multifile programs to grid engine

Reuti reuti at staff.uni-marburg.de
Wed Nov 28 22:48:57 GMT 2007


Am 28.11.2007 um 23:03 schrieb Daniel Templeton:

> When you submit a binary, all you're submitting to the qmaster is a  
> path to that binary, and hence all that the qmaster passes on to  
> the execution daemon is the path to the binary.  All the execution  
> daemon will do is (more or less) fork and exec your binary path.   
> That means that the binary path must be resolvable on the execution  
> host, and all shared libraries must be accessible and available  
> from the shared library path (or run path).
>
> You have multiple options for setting environment variables in your  
> job's environment.  The first and most obvious is the submitting  
> user's shell rc file(s).  Other favorites are the -V and -v switches.

Well, it's a matter of personal taste: I always try to avoid using - 
V/-v to the qsub (at least for $PATH and $LD_LIBRARY_PATH), as due to  
some settings in the shell the jobs might run or not (if you just  
changed your environment to execute another porgram or use other  
libs). The way I provide scripts to my users is to include all  
necessary $PATH/$LD_LIBRARAY_PATH settings in the jobscript. Whatever  
they were just testing in their interactive session, the job will  
always behave in the same way.

At some point in the past we also relied on -V, but as outlined: it's  
not easy to debug, if the job sometimes runs and sometimes not.

Only exception is the -V in the qrsh wrapper in combination with - 
inherit, so all tasks of a parallel job have the same environment.

-- Reuti


> Daniel
>
> Bruce Rothermal wrote:
>> Hi
>>
>> Can anyone point out how you would submit a binary job where the  
>> executable relies on linked libraries. Would every system on the  
>> grid have to setup with these files. Would a share file system  
>> need to be used or is Grid Engine capable of figuring out the  
>> dependencies and providing the libraries to the compute machine  
>> chosen.
>>
>> In this same line how best are environment variable handled?
>>
>> Thanks for any help on this.
>>
>> Bruce Rothermal
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list