[GE users] restriction on stdout

Reuti reuti at staff.uni-marburg.de
Sat Nov 1 23:06:14 GMT 2008


Am 01.11.2008 um 19:49 schrieb Mag Gam:

> I have noticed some unusual behavior when I pipe data to stdout and it
> gets chopped off in an array job. For example:
>
> $ wc -l file.txt
> 199971 file.txt
>
> $ cat file.txt | split --lines =50137 -a 4 -d - splitted1
>
> When I run this normally I get the correct portions.
>
> However, when I run it using qsub, in a script one of the files is
> completly different. Its missing close to 40% of the contents.

I would assume it's an NFS problem. Is it run on an NFS client? If  
possible, such task should be directed to the file server (if it's  
possible to use it for such special types of jobs - less network  
traffic, faster execution).

On the other hand: why are you using cat with a pipe? You can save a  
process with:

$ split --lines 50137 -a 4 -d file.txt splitted1

-- Reuti


> Also, the " - " means stdout in the split command. Has anyone seen  
> this before?
>
> TIA
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list