|Version 69 (modified by dlove, 14 months ago) (diff)|
Welcome to the Son of Grid Engine project
|Releases||Release notes||More information|
The Son of Grid Engine is a community project to continue Sun's old gridengine free software project that used to live at http://gridengine.sunsource.net after Oracle shut down the site and stopped contributing code. (Univa now own the copyright — see below.) It will maintain copies of as much as possible/useful from the old site.
The idea is to encourage sharing, in the spirit of the original project, informed by long experience of free software projects and scientific computing support. Please contribute, and share code or ideas for improvement, especially any ideas for encouraging contribution.
This effort precedes Univa taking over gridengine maintenance and subsequently apparently making it entirely proprietary, rather than the originally-promised ‘open core’. What's here was originally based on Univa's free code and was intended to be fed into that.
See also the gridengine.org site, which now only seems useful for the mail lists hosted there. The gridengine.org users list is probably the best one to use for general gridengine discussions and questions which aren't specific to this project.
Currently most information you find for the gridengine v6.2u5 release will apply to this effort, but the non-free documentation that used to be available from Oracle has been expurgated and no-one has the time/interest to replace it. See also Other Resources, particularly extra information locally, and the download area.
This wiki isn't currently generally editable, but could be is people would actually like to contribute; yes it needs reorganizing and expanding. If you're a known past/current contributor to gridengine and would like to help, please get in touch for access or to make any other contributions.
- 2016-03-02: Version 8.1.9 available. Note that this changes the communication protocol due to the MUNGE support, and really should have been labelled 8.2 in hindsight — ensure you close down execds before upgrading.
- 2014-11-03: Version 8.1.8 available.
- 2014-06-01: Version 8.1.7 available.
- 2013-11-01: Version 8.1.6 available, fixing various bugs.
- 2013-09-29: Version 8.1.5 available, mainly to fix MS Windows build problems.
- 2013-08-30: Version 8.1.4 available; bug fixes and some enhancements, now with over 1000 patches since Oracle pulled the plug.
- 2013-02-27: Version 8.1.3 available; bug fixes and a few enhancements, plus Debian packaging (as an add-on).
- 2013-01: The gridengine.debian repository contains proposed new packaging for Debian (as opposed to the standalone packaging now in the sge repository).
There are several distinct source repositories converted from the last Sunsource CVS versions of the same name. The sge one actually only goes back to the V60 tag — i.e. not including version 5.3 — and also has some re-buildable files removed, to keep it a bit more manageable. Only the repository trunk was converted in each case, but see History. Earlier source is still in the Univa repo.
Also, some files are omitted from those in the head of the sunsource sge repo as follows; any others missing are probably a mistake unless they're noted in the change history:
- www top-level directory: Not really relevant for a distribution, although some of it can probably be converted into useful documentation for distribution later;
- review top-level directory: Just a set of tick sheets for Sun internal use;
- gep top-level directory: The portal, which may not actually be functional now, and is anyway based on a proprietary server;
- HTML documentation: This is either rebuild-able (like Javadocs) and/or probably not useful outside the gridengine web site (like HTML-ized man pages).
In other repos, .jar files have mostly been removed (especially as some of them were from unavailable copyleft source), and the directories for support of obsolete versions have been removed from nbgui.
The repositories are maintained under darcs,1 but are mirrored to Mercurial and git — see below. To clone the sge repo, get a recent version of darcs — at least version 2 and preferably 2.5 — and then run
darcs get --lazy --set-scripts-executable http://arc.liv.ac.uk/repos/darcs/sge
which will create a directory sge of the current source. The head of the source tree is actually available directly via the Repository URL links for each repository, and if you really need all the current source without a darcs or hg client, this should do the trick:
wget -r -X /repos/darcs/sge/_darcs -np http://arc.liv.ac.uk/repos/darcs/sge find arc.liv.ac.uk/repos/darcs/ -name index.html\\?\* | xargs rm
Alternatively, you can get a tarball from the snapshot directory, but snapshots currently aren't made automatically, so it may not be so up-to-date.
You can access the repos with Mercurial (hg), e.g.
hg clone http://arc.liv.ac.uk/repos/hg/sge/
(actually just an alias for the darcs repo). You can also pull with git, e.g.
git clone http://arc.liv.ac.uk/repos/git/sge
Native servers aren't currently available for hg or git, so pulling may be relatively slow. Also note that pulls with git or hg leave scripts not executable, but the initial aimk fixes that.
Since darcs is patch based, the ordering of patches (e.g. reflected in git or hg pulls) isn't necessarily related to the dates on the log messages, and release tags in the original repo aren't reflected in git or hg. The parallel "release" repo does have a linear history with at least recent release tag if you want to track actual releases.
There is currently a git mirror of the repos at gitlab.
Building the source is rather a pain, which is something to improve at high priority, although there is a spec file for RPM-based systems which may need adjusting for non-RHEL ones. With appropriate development packages installed — the list in README.BUILD below probably still needs expanding (but see #1305) — it should build on GNU/Linux in Bourne shell with something like this:
cd source sh scripts/bootstrap.sh && ./aimk
(The aimk arguments are set up to default to -system-libs -no-gui-inst in aimk.private.) To avoid Java stuff, try:
cd source sh scripts/bootstrap.sh ./aimk -no-java -no-jni && ./aimk -system-libs -no-gui-inst -no-java -no-jni
Amongst other things, the bootstrap script allows aimk to set necessary executable permissions, which will be missing if you pulled with git or hg, or didn't use --set-scripts-executable with darcs.
For further information, see README.BUILD and README.aimk in the source tree, and ignore any inappropriate references to the sunsource web site, at least — those documents need further review in the absence of a better build system. See README.distinst concerning the scripts/distinst script for making a distribution.
Bug reporting, patches, and mail lists
There is a subscribers-only general mail list firstname.lastname@example.org; see https://arc.liv.ac.uk/mailman/listinfo/sge-discuss, or subscribe by mail as usual for Mailman.
You can report bugs by mail in the convenient time-honoured fashion to email@example.com, with control available via email2trac. They are gatewayed to the issue tracker and to a list. You can subscribe to the bugs list if you like, or keep track via the RSS feed on the tickets page. You can create an account to put tickets in the tracker via a web form, or use the "tourist" account with "sge" as the password. If you are just visiting, it's probably useful to put a mail address in the report so that you can be contacted about it. If a report doesn't show up in the tracker, it has probably fallen foul of our site spam filter, sorry.
The best way to send patches is to record them in a clone darcs repository and use darcs send. Until arrangements are made for direct contributions, they will be pushed into the main repository by hand if they look reasonable. Patches against the git or hg repository mirrors are also fine, but are more likely to need merging by hand. Otherwise, you can attach normal patches made against whatever source you have (but preferably a recent release or snapshot) to tickets or just mail them (see Contact).
If you do record changes with darcs, make the patch name a useful short summary of the patch, and include the number of any ticket it fixes in the form fix #N, where N is the ticket number. If it also refers to an item N in the sunsource Issuezilla system, it's useful to include that also in the form IZN even though the imported tickets have that reference.
As well as the partial gridengine-users mail archive the original CVS repositories have been mirrored completely, along with copies of all the items in the old issue tracker — ‘Issuezilla’, sometimes referred to as ‘IZ’, and the general material from the sunsource web site. Oracle promised to donate artifacts from the old site, but never did. Most of the extra information is available via the page of extra information.
The active IZ issues have been transferred to the issue tracker here, though some have somewhat suffered in the process; unfortunately I only discovered after the fact that they were accessible as an XML dump. MarkMail appears to have complete archives of the gridengine mail lists currently online, though with a less than ideal interface, and Gmane has gridengine-users from 2006 onwards via HTTP and NTP, although it is quite fragmentary apart from more recent items. We have a partial archive of the old -users and -dev lists (but mixed up, non-threaded, and currently not indexed). Beware of referring to items in that archive, as the URLs won't be stable in the absence of complete archives from the old list.
Copyright and Naming
The name ‘Son of Grid Engine’ was chosen for several reasons:
- Legal concerns originally over the licence terms with respect to naming (see below);
- The need for a hacker-style punny name as for GNU and several of its re-implementations (with GNU as a precedent for the first point);
- Ability to retain ‘SGE’ as an abbreviation by dropping of an insignificant word.
The Sunsource material has been copied according to the original sunsource terms. Material here has an explicit or implicit copyright under the same terms, i.e. code with no explicit terms is covered by the SISSL, and other (non-code) contributions have a permissive licence per the sunsource terms. Any explicit licence must be a free software or free documentation one, compatible with any material it modifies. The original source code is obviously under the original (rather odd!) SISSL, except for some more-or-less ‘third party’ stuff like qmake and various library code. See the licences README for a non-lawyer's summary.
The appendix to the SISSL requires a conformance test to use names like ‘Grid Engine’ for a modified version, but this appears not to be relevant if we satisfy the publishing source provisions of §3.1, according to legal advice obtained by and relayed by Univa, so the original idea of a 6.2u5 compatibility mode has been dropped.
Univa wouldn't say whether they have any relevant trademark interests, so care has been taken to avoid possible issues with that.
Note that, contrary to multiple claims, there is no evidence that any Sun (or other) material here is being distributed without an appropriate licence allowing it. See the rebuttal. Reports of licensing bugs are very welcome, of course.
As well as Univa's now-dead public repo, there was another project, Open Grid Scheduler, or ‘gridscheduler’, which unfortunately wasn't interested in community cooperation and is apparently also dead. This is a superset of their functionality, though the code may be different in parts, e.g. more-or-less cosmetic differences in OGS and stripping of Univa-specific stuff.
- There's a page of extra information, including updated versions of the howto documents from sunsource, HTML-formatted manpages, and Javadocs.
For problems with this site or queries that aren't appropriate for the list, contact Dave Love, d.love(SGE) @ liverpool.ac.uk, or you can open a ticket in the issue tracker against component "hosting". In case our spam-blocking appears to defeat you, I'm also fx(SGE) @ gnu.org. — Happy hacking
- 1. Darcs is a nice distributed version control system, but doesn't currently scale so well with a repository of this size and age. It's fine for looking at the change history and recording new changes, but currently slow at producing diffs for previous changes, for instance.