[GE users] Altering array task range does not work

reuti reuti at staff.uni-marburg.de
Wed Mar 24 10:59:04 GMT 2010

Am 24.03.2010 um 02:39 schrieb iadzhubey:

> On Tuesday 23 March 2010 04:14:38 pm reuti wrote:
>> Am 23.03.2010 um 20:07 schrieb iadzhubey:
>>> <snip>
>>> The approach I describe above uses this idea, and you can even have
>>> your array
>>> job(s) at any position inside a pipe as long as its task range is
>>> adjusted
>>> properly by the script run at the previous stage. I had to add all
>>> of my
>>> execution nodes to the submit nodes list however, since qdel command
>>> refuses
>>> to run on non-execute nodes (but why? qdel does not "execute"
>>> anything so
>>> this is counterintuitive).
>> You could setup hostbased login from all nodes to your master node and
>> issue:
>> ssh master /usr/sge/bin/lx24-amd64/qdel $JOB_ID
>> or alike in your jobscript avoiding making all nodes submit hosts.
> Yes, this is another possibility (and ROCKS even have cluster utilities for 
> that).

There is already an RFE to have a subset of commands also running on a node and disregarding the $JOB_ID, as it would then only work for the job itself.

> Still, I think more flexibility in gridengine would be nice to have, 
> like dynamic variable substitution in a wrapper script's #$ parameters for 
> instance.

This is a problem of the bash and not of SGE. For the bash lines starting with a # are ignored and no substitution will be perfomed. In fact: it would be necessary to replace them at submit time, not at execution time.

We use a wrapper for a here document to generate a jobscript with environment variable substitution on-the-fly. Maybe you can get the idea from this snippet:

cat > /tmp/jobscript.$$.sh <<EOF
qsub /tmp/jobscript.$$.sh
rm /tmp/jobscript.$$.sh

As long as EOF is not put into quotation marks, variable substitution will take place inside the here document. I even write it as:

cat > /tmp/jobscript.$$.sh <<-EOF

which will remove leading tabs and so the here document can be written with indention.

-- Reuti


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list