[GE users] specific process on specific node ( host )

Charu Chaubal Charu.Chaubal at Sun.COM
Thu May 27 14:55:19 BST 2004


On May 26, 2004, at 11:43 PM, John Hearns wrote:

> On Thu, 2004-05-27 at 03:12, Jonathan Zweig wrote:
>> Hello,
>> You can specify lists of queues to qsub; BUT sge will choose the least
>> loaded from the list you supply, so this doesn't solve my problem.  
>> My issue
>> is that I have a set of jobs which run in parallel, and I want some 
>> of them
>> to run on some machines, and other parts of the parallel system to 
>> run on
>> other machines.  I think we're talking about two different things.
>
> I think we're getting confused here with the meaning of 'parallel',
> as in 'mpich programs' and 'jobs which run at the same time'.
> I have probably misunderstood!
>
>
> I think choosing specific hosts won't scale.
> If you add new hosts to the cluster, you'll have to hard-wire in to
> a machines file again.
> OK, that may not concern you.
>
> Perhaps its better to extend the hosts complex with a machine type.
> I'm assuming here you have different types of machine - eg. fast/slow
> or big memory machines?
> then specify that machine type in qsub for each type of job.
> Or am I way off base here?
>

I guess this doesn't address the problem because the wish is to submit 
multiple tasks simultaneously on different kinds of hosts.  The current 
PE framework only envisions submitting multiple tasks on similar or 
identical hosts --- there's no real way to say something like "task 1-4 
on hosttype A and task 5-8 on hosttype B".

The only workaround I can think of is to have multiple PE with similar 
names which consists of groupings of type A and type B, and use 
wildcard in the PE request.  eg:
mype1: hostA1, hostA2, hostB1, hostB2
mype2: hostA3, hostA4, hostB3, hostB4
<etc>

Then when you submit a job, use "qsub -pe mype* 4 myjob.sh".  Then 
you'll at least get all the host types that you want.  The only thing 
is, your job script and/or start_proc_args would need to determine 
which process starts on which host, eg determine if you are on a hostA 
machine or a hostB machine, since SGE doesn't tell you this info 
directly.

Regards,
	Charu


>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>
########################################################
# Charu V. Chaubal				# Phone: (650) 786-7672 (x87672)
# Grid Computing Technologist	# Fax:   (650) 786-4591
# Sun Microsystems, Inc.			# Email: charu.chaubal at sun.com
########################################################




More information about the gridengine-users mailing list