Opened 9 years ago
Closed 9 years ago
#1420 closed defect (fixed)
qmake 8.1.0 puts "\n" in multiline commands, causing the commands to fail.
Reported by: | sbridgett | Owned by: | sbridgett |
---|---|---|---|
Priority: | normal | Milestone: | |
Component: | sge | Version: | 8.1.0 |
Severity: | major | Keywords: | multiline commands makefile qmake |
Cc: | stephen.bridgett@… |
Description
When there are multi-line commands (with the "\" continuation character at end of each line) in a "Makefile", qmake 8.0 and latest 8.1.0 replace the "\newline" with "\n", causing the command to fail. (where "newline" means an actual newline character, and "n" means the character "n").
Whereas the earlier Grid engine GE 6.1u6 replaced the "\newline" with a space so the command succeeded.
For example:
The "Makefile" contains:
all: [Tab]echo Hello \ how \ are \ you?; \ ls \ -al
where "[Tab]" is a Horizontal Tab character, and the lines below it start with spaces rather than tabs.
The "runqmake.sh" contains:
#!/bin/bash # Print the version of Grid engine: qsub -help | grep GE qmake -cwd -v PATH -- -j 2
Then submitting with qsub using:
qsub -cwd -v PATH -sync yes runqmake.sh
In the resulting "runqmake.sh.o281786" the earlier grid engine GE 6.1 outputs:
GE 6.1u6 echo Hello \ how \ are \ you?; \ ls \ -al dynamic mode Hello how are you? total 64 drwxrwxr-x 2 sbridget biology 4096 Jun 14 17:24 . drwxrwxr-x 33 root biology 16384 Jun 14 17:05 .. -rw-rw-r-- 1 sbridget biology 89 Jun 14 17:23 Makefile -rwxrwxr-x 1 sbridget biology 98 Jun 14 17:19 runqmake.sh
Whereas the latest SGE 8.1.0 outputs:
SGE 8.1.0 echo Hello \ how \ are \ you?; \ ls \ -al dynamic mode Hello \n how \n are \n you?
NOTE: The "\n" embedded within the "Hello \n how \ are \n you?"
And the "ls -al" command fails, outputting a file "runqmake.sh.e9188" containing:
/bin/sh: \n: command not found qmake: *** [all] Error 127
Which I think is because the ls command is being output as "ls \n -al"
rather than the "ls -al", so the error message "\n: command not found"
is given.
I've looked into the qmake source code, including the read.c and misc.c
files, but am not just sure why this is occurring.
Perhaps there is something I have set incorrectly in the sge configuration?
We are running SGE 8.1.0 on Centos 6 (whereas the GE 6.1u is running in Centos 5)
Thank you for help with this.
If I just use "make" on the Centos 6 cluster (where SGE 8.1.0 is installed), I get the
correct output:
make echo Hello \ how \ are \ you?; \ ls \ -al Hello how are you? total 259K drwxrwxr-x 2 sbridget biology 545 Jun 14 17:23 . drwxrwxr-x 25 sbridget biology 2414 Jun 14 11:37 .. -rw-rw-r-- 1 sbridget biology 91 Jun 14 17:23 Makefile -rw-rw-r-- 1 sbridget biology 98 Jun 14 17:20 runqmake.sh
Several of the perl scripts that we use to run processes have long multiline commands with many command-line options, so these programs fail on SGE 8.1.0.
Attachments (2)
Change History (5)
Changed 9 years ago by sbridgett
comment:1 Changed 9 years ago by sbridgett
Alternatively if I run "qmake" directly (ie. instead of the "qsub ... runqmake.sh" used above), with the parameters: "-cwd -v PATH --" then the same error occurs as shown below:
$ qmake -cwd -v PATH -- echo Hello \ how \ are \ you?; \ ls \ -al dynamic mode Hello \n how \n are \n you? /bin/sh: \n: command not found qmake: *** [all] Error 127
If I only run "qmake", then the correct output is given (but this just runs make locally, and doesn't submit job to grid engine):
$ qmake echo Hello \ how \ are \ you?; \ ls \ -al Hello how are you? total 259K drwxrwxr-x 2 sbridget biology 545 Jun 14 17:23 . drwxrwxr-x 25 sbridget biology 2414 Jun 14 11:37 .. -rw-rw-r-- 1 sbridget biology 91 Jun 14 17:23 Makefile -rw-rw-r-- 1 sbridget biology 98 Jun 14 17:20 runqmake.sh
I'm using the Linux AMD 64, ("lx-amd64") SGE 8.10 version, with source code compiled locally, rather than the rpm.
comment:4 Changed 9 years ago by dlove
SGE <sge-bugs@…> writes:
When there are multi-line commands (with the "\" continuation character at
end of each line) in a "Makefile", qmake 8.0 and latest 8.1.0 replace the
"\newline" with "\n", causing the command to fail. (where "newline" means
an actual newline character, and "n" means the character "n").
Whereas the earlier Grid engine GE 6.1u6 replaced the "\newline" with a
space so the command succeeded.
Hmm... GNU Make's previous behaviour was POSIXly wrong -- the backslash
and newline should be passed to the shell. The fix is obviously
infelicitous for qrsh, though. Output from qmake -verbose looks right,
so something seems wrong with qrsh argument handling when /bin/sh is
invoked. (Simple make commands that don't need to invoke the shell are
OK, if you can use them.)
Pending investigation, you might be able to kludge it with a
QRSH_WRAPPER (see qsub(1)) to filter the newlines or modify the argument
for the "qrsh" command in qmake/remote-sge.c/start_remote_job if you're
up for hacking it.
comment:5 Changed 9 years ago by Dave Love <d.love@…>
- Resolution set to fixed
- Status changed from new to closed
In [4265/sge]:
Makefile with multiline command.