[GE issues] [Issue 2804] New - More spooling ops than needed during job start order processing for seq jobs (classic spooling)

andreas andreas.haas at sun.com
Fri Nov 21 12:17:23 GMT 2008


http://gridengine.sunsource.net/issues/show_bug.cgi?id=2804
                 Issue #|2804
                 Summary|More spooling ops than needed during job start order p
                        |rocessing for seq jobs (classic spooling)
               Component|gridengine
                 Version|6.2
                Platform|All
                     URL|
              OS/Version|All
                  Status|NEW
       Status whiteboard|
                Keywords|
              Resolution|
              Issue type|DEFECT
                Priority|P3
            Subcomponent|qmaster
             Assigned to|ernst
             Reported by|andreas






------- Additional comments from andreas at sunsource.net Fri Nov 21 04:17:19 -0800 2008 -------
More spooling operations than needed are performed during processing of job
start orders for sequential jobs.

Reason is that the state transition from 'pending' to 'running' is logically
(i.e. inside the code) split in two parts even though it is not more than a
single operation pysically (i.e. on disk) whereas the second proposition is true
only for sequential jobs in classic spooling mode.

By unifying the logical split state transition it is possible to reduce the
spooling operations from two to one!

Expectation is that cluster utilization would benefit from this in particular
with a high throughput of sequential jobs. 

As of now the usual schema of operations during delivery of job is like this:

- enroll task from job 
- spool changes in job data structure
- do checks and preparations in task data structure
- send job
- on failure: revert changes done in task data structures and spool the task 
- on succuss: keep all changes and spool the task

This makes two operations for sequential jobs since job and task are identical
on disk in classic spooling mode.

The following cleanup facilitates to reduce the spooling operations by one

- enroll task from job 
- do checks and preparations in task data structure
- send job
- on failure: revert changes done in task data structures
- on succuss: keep all changes and spool job and task

Reason is that job and task of sequential jobs are identical on disk (classic
spooling).

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=36&dsMessageId=89346

To unsubscribe from this discussion, e-mail: [issues-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list