[GE users] run job only on hosts where software X is installed?
reuti at staff.uni-marburg.de
Mon Nov 10 18:35:44 GMT 2008
Am 10.11.2008 um 18:58 schrieb udo waechter:
> Hello sge users.
> I am pretty new to SGE and have stumbled upon a problem that I can not
> I guess I simply do not know how to search for the problem. If it was
> already asked, please excuse my incompetence.
> I would like to execute a job only an those hosts that have a certain
> piece of software installed.
> As I understand the whole problem so far, it seems to me that
> something like a load_sensor would do the trick.
> If I want to execute mencoder only on those hosts that have /usr/bin/
> mencoder installed, I could write a sensor which checks whether or not
> it is installed. But what if I would want to have this facility
> generalized? Namely: Let jobs have checks for arbitrary software in
> order to be executed on certain hosts?
in principle it could be done by a load sensor, but it this would
mean to have this "reporting facility" running all the time on the
nodes - and as the value will not change so often, it's unnecessary.
"How to look for certain packages" depends on your installations. The
load sensor could just check in some places. I think, "which myapp"
can't be used, as the PATH is not updated for an already running load
sensor. So it might miss newly installed software, unless links to
the apps are put in a general location like /usr/bin.
> I do not really get the trick with resources.
> For example, http://gridengine.sunsource.net/howto/consumable.html
> describes the 'verilog' complex, where each execution host defines how
> big its value is. Although 'complexes' would be the right thing for my
> use-case, they do not seem to be appropriate.
> Back to the 'load_sensors': Would it be possible (if yes, how) to have
> a parametrized load_sensor?
Using resources is the right approach. You will just need to:
1 - create a new entry with "qconf -mc" and a line like
myapp ma BOOL == YES
NO 0 0
2a - then attach this either to a queue "qconf -mq my_queue" by
hostgroup or individual host:
2b - or each host "qconf -me my_host":
3 - submit the jobs and request this resource:
qsub -l myapp my_job.sh
(for boolean resources the attribute "TRUE" is assumed by default,
but you could also use "-l myapp=TRUE"). Whether you implement 2a or
2b is for this intended setup personal taste and depends more of the
overall queue and host organization, i.e. which is less work to set
it up would be to attach it to a queue; OTOH more logical would be to
attach it to each host, as this will also work if there are several
queues on each machine and it's a feature of a particular machine. In
case of 2b you could also use a cron job to update the flag once a
day after a similar search like you intend with the load sensor.
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users