[GE users] intensive job

Mag Gam magawake at gmail.com
Sat Oct 25 02:05:53 BST 2008


    [ The following text is in the "UTF-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Sure, I can output. But how can I keep state though? How can I
distinguish between first and second and third, etc...output?



On Fri, Oct 24, 2008 at 8:44 PM, leeping <leeping at mit.edu> wrote:
> Can you output to ten different files and then concatenate them when the job
> is done?  That way you would preserve the order.
>
> Mag Gam wrote:
>>
>> Hello All.
>>
>> We have a professor who is notorious for bring down our engineering
>> GRID (64 servers) servers due to his direct numerical simulations. He
>> basically runs a Java program with -Xmx 40000m (40 gigs). This
>> preallocates 40 gigs of memory and then crashes the box because there
>> are other processes running on the box. Each box has 128G of Physical
>> memory. He runs the application like this:
>> cat series | java -Xmx 40000m fluid0 > out.dat
>>
>> the "series" file has over 10 million records.
>>
>> I was thinking of something like this: split the 10 million records
>> into 10 files (each file has 1 million record), submit 10 array jobs,
>> and then output to out.dat. But the order for 'out.dat' matters! I
>> would like to run these 10 jobs independently, but how can I maintain
>> order?  Or is there a better way to do this?
>>
>> By him submitting his current job it would not be wise...
>>
>> TIA
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list