1

Here's the set up:

  • Two Ubuntu 9.10 servers - LAMP environment (not really an option to switch OS)
  • One MD3000i with iscsi connection

Here's what I'd like to do:

  • Have both server see the files that the other one writes to the MD3000i without unmounting the drive. My connections are fine to the MD3000i (using open-iscsi). The MD3000i is great in almost every way I need except for one listed below.

Here's what I don't want to do (for reasons I'm not going to go into - been down this road already):

  • Use NFS mounts

Example:

  • Server1.com and Server2.com write to the same drive on the MD3000i. The folder that they are both mounted to is labeled /mnt/files/. They both recognize the partition size correctly.

    Then Server1.com writes test1.1.txt to /mnt/files and Server2.com writes test1.2.txt to /mnt/files. Files are written during ssh connections directly on server in this situation. Server1.com does not see test1.2.txt without remount. Server2.com does not see test1.1.txt without remount.

    Oddly, if I write the files from an apache server via PHP and the file has a unique name, both server's see it! But if the name is not unique I then have an I/O error (no surprise).

    I'm familiar with gfs, gfs2 and ocfs2 but far from competent. I've tried various methods for formatting the drive and mounting it. It seems as though I just can not get the recipe correct.

Question:

What are your suggestions? I've tried many different ways to set up what I'm been lead to believe is the holy grail - clustering - with no real success. I know what doesn't work. Could really really use a boost from some helpful tips.

Improve this question
6
  • This is a bad idea, like having 2 concurrent windows installations accessing the same NTFS drive. Each believes it is in control of the file allocation tables and the journaling system, and they conflict with each other. NFS allows one to be the host... What if you added a 3rd server (just a commodity box), to serve as a storage area network controller?
    – Fosco
    Aug 5, 2010 at 19:46
  • @Fosco - This is a LAMP environment AND I stated that I am not interested in NFS mounts (which I would pipe through one server anyway).
    – Patrick R
    Aug 5, 2010 at 20:17
  • +1 There is a lot of information about different systems but it is difficult to find real life experiences documented. This is a great question.
    – Richard Holloway
    Aug 5, 2010 at 22:20
  • You say "not NFS" but don't really give a valid reason why you wouldn't use it. Is it purely because the MD3000i is just giving out iscsi?
    – dotwaffle
    Aug 5, 2010 at 23:49
  • @dotwaffle - Nothing is gained by my providing extensive documentation for not using NFS. My question states that I'm looking for another solution - been down that road - will say that I was burn by it's slowness in my environment.
    – Patrick R
    Aug 6, 2010 at 12:50

2 Answers 2

Reset to default
1

Short answer : GlusterFS

Long Answer: I have set up a system with DRDB which I thought would be the ideal solution as it is well documented and loads of examples can be found of running LAMP on it.

But it isn't the ideal solution as you only have the file system mounted on one box at a time and you have to handle a lot of services starting etc when one machine fails over to the other. Also it is only for two machines.

I built a test glusterfs cluster of two virtual Ubuntu Lucid Lynx machines in virtualbox today and not only is it ridiculously easy to set up, everything you need is in the repos.

From my tests so far it appears to be everything I want.

It is a bit like a two way (or actually multiway) nfs. You can have as many nodes as you want accessing it and the file system handles the locks etc, so ignore the "This is a bad idea, like having 2 concurrent windows installations accessing the same NTFS drive..." comment above, because that is way off course.

Make sure you get the server - client aspect of it! I missed the point of the FUSE mount until I considered the file locking.

I have yet to test it under heavy MySQL load but so far I am impressed. My web searches had found very little to suggest GlusterFS is up to the job so I am reassured by Tom O'Conner comment too.

If I find any more relevant information tomorrow I will post back.

Improve this answer
3
  • @Holloway - +1 - two way nfs? If GlusterFS can provide me with mutliway to a single partition then I'm certainly going to start experimenting with it. I wonder what kind of read/write speeds I'm going to experience. That's my biggest hang up. I need to transfer my files over multiple iscsi connections on LAN to shared drive but do not, DO NOT, want to risk doing this through a single NFS mount on a third LAMP server.
    – Patrick R
    Aug 6, 2010 at 12:58
  • I am stuck on actually getting mysqld to run using the mounted glusterfs as the datadir. BUT for files it works great. I am not sure if you are using the replicated filesystem for MySQL databases or just files. I am still testing so not too sure on speed over the wider internet, and still have to look at how to secure the traffic. Also ,might be looking into MongoDB soon as an alternative to MySQL. I would be interested to hear how you get on with your research too.
    – Richard Holloway
    Aug 6, 2010 at 14:50
  • @Holloway - Perfect suggestion! If anyone else is interested, there is a very straightforward guide at: howtoforge.com/…. The wget links are out of date but you'll be able to find the up-to-date files based on those links.
    – Patrick R
    Aug 10, 2010 at 17:44
0

Ok ... you excluded NFS. something similar could be AFS.

Or you could also try sshfs.

I don't know if they solve your problems, but they it's good to have them in mind :>

Improve this answer
2
  • sshfs is a bit slow... hell of an overhead. I recommend GlusterFS. I've used it in a LAMP environment, in production to great effect.
    – Tom O'Connor
    Aug 5, 2010 at 20:38
  • I've heard of GlusterFS and wasn't sure if I wanted to go down that road. I'll take a look at it again.
    – Patrick R
    Aug 5, 2010 at 21:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .