[GE users] Altering array task range does not work

iadzhubey iadzhubey at rics.bwh.harvard.edu
Tue Mar 23 16:31:09 GMT 2010


Hi Reuti,

So you are saying '-t' option (task range) cannot be modified with qalter 
command? The qalter manpage seems to disagree so at least this should be fixed 
in the documentation and the (rather odd) syntax deserves documenting too.

It's a pity if this can't be done. As I said, I found a reasonably simple 
workaround that works for my purpose but it's a rather awkward one. The reason 
I need this dynamic adjustment of the task range is that I wrote a program 
that automatically generates and submits "pipelines" of jobs based on simple 
template description. Pipelines are common in many applications: output of one 
program is fed into the next one and so on, just like Unix "pipes". With my 
script, a user can implement his/her new analysis pipeline in less than 5 
minutes with no programming involved. However, the input of a program in the 
middle of a pipeline is not defined until the previous one that generates it 
has completed. If this is an array job we simply don't know how many tasks it 
will comprise until we have an output file from the previous stage of a pipe 
available. Hence, the size of this array job should be determined dynamically, 
right before it is actually *executed*. It can't be specified correctly at the 
point when the job is *submitted*.

I would suggest this as a feature request for the next gridengine version.

--Ivan

On Tuesday 23 March 2010 04:29:38 am reuti wrote:
> Am 23.03.2010 um 08:56 schrieb dougalb:
> > Can you provide the full command you are using?
> > 
> >  $ qalter 4562 -t 1000-2000 -h s
> 
> Great, thx for clarifying this. But I fear it's not the answer the
> original poster is expecting to change the complete range of the
> alread submitted jobs as long as they are in "qw" state.
> 
> ==
> 
> Interesting syntax - not easy to find. It also doesn't correspond with
> the manpage. It should be mentioned there:
> 
> The -t option with `qalter` is only valid for the -h {u|s|...} option,
> the order of these two options doesn't matter and a wc_job_range_list
> can be disregarded. In fact it will be used independently. I mean:
> 
> $ qalter -h u 917 -t 15-25 918
> 
> is valid to change two jobs. The funny thing is:
> 
> $ qalter -A sdffd 917 -t 15-25 -h n
> 
> is allowed and changes the hold (subrange) and account (for all tasks
> of course), while:
> 
> $ qalter -A sdffd 917 -t 15-25
> 
> throws an error. No -t allowed w/o -h, hence it's really an option to -
> h.
> 
> -- Reuti
> 
> > That should work.
> > 
> > On Mon, Mar 22, 2010 at 10:37 PM, iadzhubey
> > 
> > <iadzhubey at rics.bwh.harvard.edu> wrote:
> >> Any suggestions at all? Anyone, please?
> >> 
> >> --Ivan
> >> 
> >> On Monday 22 March 2010 10:48:53 am iadzhubey wrote:
> >>> Hi Reuti,
> >>> 
> >>> On Monday 22 March 2010 08:42:44 am reuti wrote:
> >>>> Hi,
> >>>> 
> >>>> Am 20.03.2010 um 08:57 schrieb iadzhubey:
> >>>>> I am trying to adjust the task range of an already submitted and
> >>>>> pending array job but cannot figure out how to do this (and if
> >>>>> it's
> >>>>> even supported). Qalter manpage lists '-t' as supported option
> >>>>> but when
> >>>>> specified with qalter command it is recognised as an (incomplete)
> >>>>> wc_job_range_list specification instead, which results in a syntax
> >>>>> error message.
> >>>> 
> >>>> you mean the error message like:
> >>>> 
> >>>> found lonely '-t 15-20' option (The -t option needs a leading job
> >>>> name)'?
> >>> 
> >>> Yep, this is the error message. I've also tried every other
> >>> combination of
> >>> command-line switches to qalter I can think of but to no avail.
> >>> 
> >>> Any hope this may actually work? For now, I am using a crude
> >>> workaround by
> >>> submitting array jobs with the fixed maximum range (e.g., -t
> >>> 1-75000) and
> >>> then letting the first dispatched task of each array qdel all
> >>> extraneous
> >>> tasks at runtime. This works but is not an elegant solution.
> >>> 
> >>> Best,
> >>> Ivan
> >> 
> >> The information in this e-mail is intended only for the person to
> >> whom it is
> >> addressed. If you believe this e-mail was sent to you in error and
> >> the e-mail
> >> contains patient information, please contact the Partners
> >> Compliance HelpLine at
> >> http://www.partners.org/complianceline . If the e-mail was sent to
> >> you in error
> >> but does not contain patient information, please contact the sender
> >> and properly
> >> dispose of the e-mail.
> >> 
> >> ------------------------------------------------------
> >> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessage
> >> Id=250613
> >> 
> >> To unsubscribe from this discussion, e-mail:
> >> [users-unsubscribe at gridengine.sunsource.net ].
> > 
> > ------------------------------------------------------
> > http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageI
> > d=250711
> > 
> > To unsubscribe from this discussion, e-mail:
> > [users-unsubscribe at gridengine.sunsource.net ].
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=
> 250715
> 
> To unsubscribe from this discussion, e-mail:
> [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=250866

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list