wiki:WikiStart

Version 24 (modified by dlove, 9 years ago) (diff)

--

Welcome to the Son of Grid Engine project

The Son of Grid Engine is a community project to continue Sun's old gridengine free software project, now that Oracle have taken it over, are not contributing their source, and may close down the Sunsource web site.

The idea is to encourage sharing, in the spirit of the original project, and informed by long experience of free software projects and scientific computing support. Please contribute, and share code or ideas for improvement, especially any ideas for encouraging contribution.

Pending adding to this site, you can look at the old gridengine site for extra documentation etc. as long as it's there. Currently anything for the gridengine v6.2u5 release will apply to this effort. (The v6.2u5 wiki documentation isn't too easily findable from there.)

This wiki isn't currently editable, but should be eventually.

News

  • 2010-11-15: URLs of the form http://arc.liv.ac.uk/repos/hg/repo and http://arc.liv.ac.uk/repos/git/repo now work for hg and git respectively.
  • 2010-11-14: You can now access the repos with Mercurial (hg) at the same URLs as with darcs, e.g. hg clone http://arc.liv.ac.uk/repos/darcs/sge/. (They should probably be aliased to avoid confusion.) You can also pull with git, but the relevant URL is the darcs one with /.git appended, e.g. git clone http://arc.liv.ac.uk/repos/darcs/sge/.git. Servers are not currently available for hg or git, so pulling may be relatively slow; also mirroring currently isn't automatic, so the hg and git repos may be somewhat out-of-date. Also note that pulls with git or hg leave scripts not executable—see the amended build instructions.
  • 2010-11-14: Most jar files have been expunged since several from the sunsource repository are copyleft, and can't be distributed as-were. Since this involved re-writing history in the arco and nbgui repos, if you pulled from them, you'll need to do it again, unfortnuately.
  • 2010-11-10: A general mail list and bug-reporting by mail are available.

Repositories/Source

There are several distinct source repositories converted from the last Sunsource CVS versions of the same name. The sge one actually only goes back to the V60 tag—i.e. not including version 5.3—and also has some re-buildable files removed, to keep it a bit more manageable. Only the repository trunk was converted in each case, but see History.

Also, some files are omitted from those in the head of the sunsource sge repo as follows; any others missing are probably a mistake unless they're noted in the change history:

  • www top-level directory: Not really relevant for a distribution, although some of it can probably be converted into useful documentation for distribution later;
  • review top-level directory: Just a set of tick sheets for Sun internal use;
  • gep top-level directory: The portal, which may not actually be functional now, and is anyway based on a proprietary server;
  • HTML documentation: This is either rebuild-able (like Javadocs) and/or probably not useful outside the gridengine web site (like HTML-ized man pages).

In other repos, .jar files have mostly been removed (especially as some of them were from copyleft source), and the directories for support of obsolete versions have been removed from nbgui. This may affect building Java sources.

The repositories are maintained under darcs,1 but are mirrored to Mercurial and git—see News. To clone the sge repo, get a recent version of darcs—at least version 2 and preferably 2.5—and then run

darcs get --lazy --set-scripts-executable http://arc.liv.ac.uk/repos/darcs/sge

which will create a directory sge of the current source. The head of the source tree is actually available directly via the Repository URL links for each repository, and if you really need all the current source without a darcs or hg client, this should do the trick:

wget -e robots=off -r -X /repos/darcs/sge/_darcs -np http://arc.liv.ac.uk/repos/darcs/sge 
find arc.liv.ac.uk/repos/darcs/ -name index.html\\?\* | xargs rm

Alternatively, you can get a tarball from the snapshot directory, but snapshots currently aren't made automatically, so it may not be so up-to-date.

Building

Building the source is currently rather a pain, which is something to improve urgently. With appropriate development packages installed—they need documenting—it should build on GNU/Linux in Bourne shell with something like:

cd source
csh aimk -only-depend && scripts/zerodepend && ./aimk depend &&
./aimk -system-libs -no-gui-inst

or, to avoid Java stuff:

cd source
csh aimk -only-depend && scripts/zerodepend && 
./aimk -no-java -no-jni depend &&
./aimk -system-libs -no-gui-inst -no-java -no-jni

The explicit use of csh the first time allows aimk to set necessary executable permissions, which will be missing if you pulled with git or hg, or didn't use --set-scripts-executable with darcs.

For further information, see README.BUILD and README.aimk in the source tree, and ignore any inappropriate references to the sunsource web site, at least—those documents need review in the absence of a better build system.

Bug reporting, patches, and mail lists

There is a subscribers-only general mail list sge-discuss@liv.ac.uk; see https://arc.liv.ac.uk/mailman/listinfo/sge-discuss, or subscribe by mail as usual for Mailman. Bug reports mailed to sge-bugs@liv.ac.uk are gatewayed to the issue tracker and to a list.

You can report bugs by mail in the convenient time-honoured fashion to sge-bugs@liv.ac.uk. They are gatewayed to the issue tracker and to a list. You can subscribe to the bugs list if you like, or keep track via the RSS feed on the tickets page. You can create an account to put tickets in the tracker via a web form, or use the "tourist" account with "sge" as the password. If you are just visiting, it's probably useful to put a mail address in the report so that you can be contacted about it. If a report doesn't show up in the tracker, it has probably fallen foul of our site spam filter, sorry.

The best way to send patches is to record them in a clone darcs repository and use darcs send. Until arrangements are made for direct contributions, they will be pushed into the main repository by hand if they look reasonable. Otherwise, you can attach normal patches to tickets or just mail them (see Contact), maybe even against the 6.2u5 source, if you have them ready-made.

If you do record changes with darcs, make the patch name a useful short summary of the patch, and include the number of any ticket it fixes in the form fix #N, where N is the ticket number. If it also refers to an item N in the sunsource Issuezilla system, it's useful to include that also in the form IZN even though the imported tickets have that reference.

History

The original CVS repo is currently still available if you need more history, and has been mirrored in case it disappears, along with copies of items in the issue-tracker (‘Issuezilla’), sometimes referred to as ‘IZ’, and the general material on the sunsource web site. The active issues have been transferred to the issue tracker here, though some have somewhat suffered in the process; I only discovered after the fact that they were accessible as an XML dump. If the Sunsource mailing list archive disappears, you can still use at least MarkMail and Gmane, but it would be good if we could stash a dump of the email history too somehow.

Concerning copying the Sunsource material, see the Sunsource terms. Material here has an explicit or implicit copyright under the same terms, i.e. code with no explicit terms is covered by a BSD licence, and other contributions have a permissive licence. Any explicit licence must be under a free software licence compatible with any material it modifies. The source code is obviously under the original (rather odd!) SISSL licence except for some essentially ‘third party’ stuff like qmake.

There's another project, Open Grid Scheduler, or ‘gridscheduler’, which is also continuing work on gridengine but has different aims. Unlike this one, its repository doesn't use a distributed revision control system, and doesn't include history from before the gridengine v6.2u5 release, or the documentation. It would be better if there was only one project, but this satisfies the need for those things missing from OGS (and more, like a bug-tracker that doesn't involve creating yet another account), which should ease contributing.

The sge source here is currently a superset of OGS's, and is likely to remain so. It includes the changes from OGS, or close equivalents, plus some important fixes for persistent or immediate crashes in some circumstances, and other less important changes, as well as documentation. It is currently more unstable than OGS, but breakage should get fixed, at least if it isn't inherited from the original code base, and there will probably be stable and development branches eventually.

Contact

For problems with this site or queries that aren't appropriate for the list, contact Dave Love, d.love(SGE) @ liverpool.ac.uk, or you can open a ticket in the issue tracker against component "hosting". In case our spam-blocking appears to defeat you, I'm also fx(SGE) @ gnu.org. — Happy hacking


  1. 1. Darcs is a nice distributed version control system, but doesn't currently scale so well with a repository of this size and age. It's fine for looking at the change history and recording new changes, but currently slow at producing diffs for previous changes, for instance.