[GE users] DRMAA output redirection oddness

reuti reuti at staff.uni-marburg.de
Wed Dec 8 13:29:29 GMT 2010

    [ The following text is in the "utf-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Am 08.12.2010 um 13:10 schrieb epistemyscott:

> On 7 Dec 2010, at 19:19, reuti wrote:
>> Am 07.12.2010 um 15:29 schrieb epistemyscott:
>>> The DRMAA bindings get upset if the output path doesn't start with a ':'. The official format in the Javadoc is "[hostname]:file_path", although I'm not specifying any hostname - I presume this means "leave things as they are", though I may be wrong - I couldn't find any explicit explanation of what happens if you don't specify a host name.
>> What about using a native specification and -o/-e/-wd for now, are the pseudo variables working there?
> The stuff I'm writing has to work with TORQUE as well (hence the DRMAA), so I'd like to stay away from native specs as much as possible. Nonetheless, I gave this a try, and it doesn't work - using job.setNativeSpecification("-wd ~/SGE-test -o SGE-test.log -e SGE-test.err.log"); and removing the DRMAA lines fails with the familiar message:

AFAICS -wd doesn't support pseudo variables. So I would assume you need the full path to /home/whoever/SGE-test

> 12/08/2010 11:54:28|worker|host123|W|job 92.1 failed on host host123.epistemy.com general changing into working directory because: 12/08/2010 11:54:27 [1024:14268]: error: can't chdir to /SGE-test: No such file or directory
>>> I've tried setting the transferFiles attribute, which is mentioned in the Javadoc, to "new FileTransferMode(false, false, false)", but that results in an IllegalArgumentException. Also, the working directory can't be changed even if I don't specify an output or error path, and the working directory has no colons in it. Perhaps you've hit on the problem for the output and error paths, but I doubt it's the problem with the working directory.
>> http://gridengine.sunsource.net/howto/filestaging/filestaging6.html
>> You defined "delegated_file_staging true"?
> No. I don't want to do delegated file staging - I want to use a shared network file system. Although it doesn't work with DRMAA, doing "qsub -wd ~/SGE-test -o test.log -e test.err.log ~/testScript.sh" on the command line works fine.

Yes, but here the shell will expand it. When you try:

$ qsub -wd "~/SGE-test" -o test.log -e test.err.log ~/testScript.sh

it should also fail. OTOH: -o "~/test.log" (not expanded by shell here!) should work, but it doesn't. It crashed the sge_execd for me in most cases. Same when you put it in the job script with:

#$ -e ~/test1 -o ~/test2

Maybe the expansion of pseudo variables is broken per se as I see in the messages file:

12/08/2010 14:24:28|  main|pc15370|E|invalid user name "/t02 at 361350267310343^_^H356q?yq?@361350267310343^_^H^P"
12/08/2010 14:24:28|  main|pc15370|E|invalid user name "/t02 at 361350267310343^_^H356q?yq?@361350267310343^_^H^P"

We use job generators and they always put full pathnames there, so it never hit us.

When you observe the same, I would suggest to file an issue about it. Maybe the DRMAA problem is only showing up a different issue.

-- Reuti


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list