1

I have a network of 20 machines, running Ubuntu 10.04.

Each machine has about 200[GB] of data that I'd like to share with all other 19 machines for READ ONLY PURPOSES. The reading should be done at the fastest possible way.

A friend told me to look into setting up HTTP / FTP. Is it indeed the optimal way to share data between the machines (better than NFS)? if so, how do I go about it?

Is there a python module that would help in accessing/reading the data?

UPDATE: Just to clarify, all I want is to be able (from within machine X) to access one of machine Ys files and LOAD IT INTO MEMORY. all of the files are of uniform size (500 [KB]). Which method is fastest (SAMBA / NFS / HTTP / FTP)?

1
  • @user540009, "Each machine has about 200[GB] of data that I'd like to share with all other 19 machines for READ ONLY PURPOSES." sounds like you have 200GB/machine. Perhaps you can change your question to reflect only 500KB.
    – Greg
    Apr 4, 2011 at 4:19

2 Answers 2

1

There are hundreds of ways to solve this problem. You can mount a FTP, or HTTP file system over fuse, or even use NFS (why not?). Search for httpfs2 or curlftpfs (or even sshfs, which should not be used if you are looking for performance)

But the problem I see is, you have a single point of failure of the one and only master machine. Why not distribute the storage?

I usually using glusterfs [1], which is fast and can be used in different modes.

[1] http://en.wikipedia.org/wiki/Distributed_file_system

0
2

With python you can start up a webserver via a simple one-liner in the directory where the data is stored.

python -m SimpleHTTPServer

Edit:

It creates a simple web server on port 8000, performance wise I cannot tell you much, and for that type of questions it would be better to ask in SuperUser and not SO.

It doesn't autostart, but it wouldn't be hard to make it.

10
  • Thank you. Can you elaborate more? how do I access it? does it remain permanent even after the computer reboots? performance vs. NFS?
    – user76976
    Apr 4, 2011 at 0:11
  • Updated my response.
    – mhitza
    Apr 4, 2011 at 0:15
  • WOW, never knew about it! Awesome!!!! +1
    – zengr
    Apr 4, 2011 at 0:16
  • @user540009 You can put this command in your .bashrc, so when your machine boots you, the server will startup automatically.
    – zengr
    Apr 4, 2011 at 0:28
  • python -m SimpleHTTPServer might be suitable as one time thing, no more.
    – jfs
    Apr 4, 2011 at 0:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .