1

I have a bunch of different binary files (total size of all files is circa ~500GB) stored in a hard drive that is mounted to machine001.

I have 200 other machines that need to access this data for READ-ONLY PURPOSES.

What is the best way to share this data with all other computers, so that READS are fastest? (I use Ubuntu Linux)

Is there a different scheme that would be even better for fastest reads?

migrated from stackoverflow.com Apr 3 '11 at 6:44

This question came from our site for professional and enthusiast programmers.

  • Should the dataset be read by random access from the 200 machines or should it be cloned there? Does the dataset ever change? – Bittrance Apr 3 '11 at 6:51
  • @Bittrance: existing data files do not change. but new ones are added. dataset be read by random access from the 200 machines. – user76976 Apr 3 '11 at 21:18
0

SAMBA is an option, but IMO isn't the fastest especially if your clients are XP. I would suggest looking into WebDAV, which Windows natively supports but you'd be better off using a 3rd party client. Also, NFS, which Windows can support by installing MS Client Services for NFS.

Protocols aside, the read speed is going to be limited to your Ubuntu machine's hardware most likely. If you can get over 125MB/s reads directly on the machine, then you're good to go as that's the limit of gigabit ethernet (assuming you have that). If you're on fast ethernet (100Mb/sec), that's going to be your bottleneck as it'll limit reads to 10MB/s.

  • all machines are running ubuntu Linux, no windows. – user76976 Apr 3 '11 at 21:19
  • 1
    Then definitely NFS. – churnd Apr 3 '11 at 23:19
0

If other machines are running Windows, Samba is the only thing that would satisfy your needs. It has plenty of configuration options, and can give you read-only access you desire.

0

I think it depends what you want to do (I mean, how the data to be read are sensitive); because 200 computers for 1 server seems a lot for me. I would tend to use some distributed stuffs, like p2p sharing system; but I've never done it on local net.

0

Depending on what's accessing the data and how you want to read it you could look into either caching (perhaps you could deliver these files by http and cache that info either locally or on nominated cache nodes).

The other solution could be to look at using some kind of distributed file system like MooseFS. There's a list of some other distributed file systems here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.