[GE users] File server overloading
landman at scalableinformatics.com
Thu Jan 31 12:58:49 GMT 2008
[ The following text is in the "ISO-8859-1" character set. ]
[ Your display is set for the "ISO-8859-10" character set. ]
[ Some special characters may be displayed incorrectly. ]
Ron Chen wrote:
> Lustre is good, I heard many good feedbacks about it. And
We have heard some fairly mixed comments. There are a few design issues
(centralized non-replicated MDS) that are at a minimum, a significant
concern to anyone wanting to make sure they have a live file system at
For some workloads Lustre is fine.
> Lustre+Gigabit Eithernet should be good enough, Infiniband is
> fast and low latency but latency is mainly needed for parallel
> computing (e.g. MPI jobs with lows of small message exchange
> between tasks).
... or very high bandwidth storage. You can get 800+ MB/s (B as in
bytes, not "b" for bits) to a node from storage using NFSoverRDMA.
Add to this now, the inexpensive IB cards and switch, and frankly it
gets difficult not to look at IB for storage systems.
> Also, pNFS (parallel NFS) is supposed to be better than
> traditional NFS too, anyone tried it in a compute farm
pNFS is sadly, not yet ready for prime time (from an implementation
view). There are bits and CVS things floating around out there.
That said, there are some "up and coming" storage software systems:
GlusterFS is interesting, and very fast. It doesn't suffer the MDS
issue of Lustre. Our testing of it has been, well, interesting.
To the original poster, GFS is known to be slow. If you need high
performance parallel file systems, you are better advised to look
elsewhere. GFS was not designed for the high throughput/scalable file
systems that people need more of today. Lustre sort of is, but it has a
few design issues that you should be wary of ... as long as you don't
mind using it for fast temporary storage, Lustre is fine. Permanent
storage is another issue. There are commercial offerings such as
Polyview, Ibrix, etc. Some of these scale ok, but will have a cost to
them. Then there are the integrated storage appliances (Isilon, DDN,
etc...). Those will scale and work well as safe and fast storage, and
scale well. But they will also cost you.
You have choices, not all of them free, but some of them are quite good.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users