[GE users] SGE Job files and directories

Amit H Kumar AHKumar at odu.edu
Tue Oct 31 18:33:18 GMT 2006



Andy Schwierskott <andy.schwierskott at sun.com> wrote on 10/31/2006 12:07:20
PM:

> Hi,
>
> to reduce spooling overhead in classic spooling we introduced the
creation
> of individual files for the task of an array job - for this it was
necessary
> to create directories under the jobs/ directory.
>
> In Unix hover a directory cannot hold more than 2^16 subdirectories :-(.
> This made it necessary to create a more complex directory structure that
a
> site can have more than 2^16 running and pending jobs.

If I understand this right, this hover is a limitation of GUI features.
Please correct me if I am wrong.
In that case this limitation was probably a concern for sites doing large
scale HPC.

>
> What you observe is just an internal implementation detail which is of
> course completely solved wiht BDB spooling. It does not reflect parallel
> jobs.

Thank you for feedback,
-AK


>
> Regards,
> (SGE-)Andy
>
> > Hi SGE,
> >
> > I noticed that for jobs submitted to more than one node/slot, there is
a
> > directory created as opposed to jobs submitted to only one node/slot.
> > Under the following directory path on the head node.
> > /opt/gridengine/default/spool/qmaster/jobs/00/0000
> >
> > Curious to know, What is the idea behind this ?
> >
> > Also the "common" file in the directories created differ from the
"common"
> > file under the sub-directory
> > #diff /opt/gridengine/default/spool/qmaster/jobs/00/0000/0841/common
> > /opt/gridengine/default/spool/qmaster/jobs/00/0000/0841/1-4096/1/common
> >
> > Are these files feeding information to Grid Engine specific to
individual
> > hosts ?
> >
> > Thank you for any comments,
> > -AK
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list