1

So I've been trying to create a Dropbox like system to allow multiple EC2 instances to work with the same files. I'm writing the programs that actually modify the files myself, and I'm not worried about issues with concurrent modification errors, as I've designed my way around it. Now the closest I could find to do something like this was to use NFS which seems like a ray of sunshine from heaven. (And it didn't look that complex either!) Now, I ALMOST got it all set up, and then I ran headfirst int a brick wall. I've been following the following tutorial: http://hunterford.me/amazon-ec2-and-nfs/

The only thing I did differently was to replace this line:

/var/www/test   ec2-180-71-131-129.compute-1.amazonaws.com(rw,async)

with this line:

/home/ec2-user/Players *(rw,async)

The first part I changed because that's where I wanted to put my files that I wanted to share. The second part I changed because I couldn't figure out what I was supposed to put there, and he said I could put a wildcard. (I know all of you are cringing right now, I was too) I didn't get an error, so I went with it. Everything else went smoothly, and I did set up the security groups with all the ports listed opened.

When I went to mount the file on my other machine:

mount -t nfs {MY_FIRST_MACHINES_DNS_HERE}:/home/ec2-user/Players /home-ec2-user/Players

Now, first it sat there for a while just blinking at me. Eventually it timed out. Does anyone see what I did wrong? Frankly it could be anything because I have no idea what I'm doing when it comes to NFS. I've been using the standard (Free Tier) Amazon Linux AMI, with 16GB of the General Purpose SSD storage, on a t1.micro instance.

  • cough cough... Bump. – Andrew Gies Aug 14 '14 at 0:03
1

As of 2017 the easiest way to do this is AWS Elastic File System (EFS), which is NFS v4.1 compatible. You pay $0.30 to $0.36 per GB stored, with no pre-allocation of disk, so if you delete files you immediately pay less. It's not available in every region.

Latency is apparently higher than EBS, enough so that it can add measurable latency for some use cases, such as web hosting. Throughput is meant to be very good. I expect AWS are working on improvements. Performance testing is suggested to ensure it meets your needs.

| improve this answer | |
0

Try changing the nfs in the mount command to nfs4. NFS v3 and earlier depend on many daemons and port mappings in order to work correctly. NFSv4 just needs the nfsd and port 2049/tcp.

About the *, just substitute it with the ip address of the client instances. If you have more than 1, just add them like:

/mount/point ip1(options1) ip2(options2) ...

| improve this answer | |
  • Great, thanks. I'll test this the next time I have a chance to be on the server, which will be tomorrow. When I'm installing NFS with the yum command, do I need to change anything for NFSv4 to work, or is everything else the same? – Andrew Gies Aug 10 '14 at 4:44
  • The standard nfs server packages allow for up to v4 since RH6 (RH5 too but i think it needed some tweaking) – Gheesh Aug 10 '14 at 9:44
  • Ok, I tried that. Ran it in verbose mode and all I got was a series of trying text-based options then it lists the (correct) IP's for my server/client, then it says mount(2): connection refused. Eventually it gives up and times out. This smells like something with the ports to me. – Andrew Gies Aug 10 '14 at 19:11
  • That was trying it with the * in the /etc/exports still. If I replace it with my client Public/Private DNS / Public/Private IP, when I run exportfs -ar, I get an error: {Client DNS/IP Here}:{MY-FILE-LOCATION-ON-SERVER}: No such file or directory. My security groups allow UDP 111, 32770 - 32800, and TCP 111, 2049 on both client and sever. – Andrew Gies Aug 10 '14 at 19:18
  • EDIT: Per the note at the bottom of the tutorial I referenced, I also opened up UDP 32806, which resulted in the same errors on mount. – Andrew Gies Aug 10 '14 at 19:28
0

Keep in mind that, if you do use EFS, your throughput is based on the overall size of your EFS volume, with a burst credit allocation. Think of it as being the same resource allocation model that you're using with those t1 or t2 EC2 instances - you're paying less for the instance, and you're getting an allocation of burst usage. If you run out of those credits, your resources get lower priority. Your t2 instance "feels" slower, your EFS file transfers take longer to start and finish.

If this happens in EC2, you can bump your instance flavor up to basically anything that doesn't start with the letter t.

In EFS, as of this writing, it seems that your only option is to inflate the size of your filesystem.

If performance isn't as important to you as availability and redundancy, EFS may still be a fit for you. Run some tests if you have any doubts.

As of this writing, EFS gives you 50MB/sec per TB (or fraction) per filesystem, bursting to 100MB/sec as long as you have the credits. The throughput is shared across all instances accessing that EFS filesystem. Ten instances accessing a 1TB filesystem? 5 to 10 MB/sec each.

References:

Amazon EFS FAQ (see the question on "How much throughput can each filesystem support?"): GitLab's advice on hosting repos on EFS (tl;dr - "Don't.")

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.