0

I would like to create "infinite" directory in terms of storage in an EC2 instance. Is it possible to symlink directories to S3 bucket?

I did not find any solution in the Internet, maybe somebody knows?

2 Answers 2

4

Amazon S3 is not a file system. You cannot just mount it to your EC2 instance like a regular disk or EBS. It now follows that a symlink is not possible.

You can however use s3fs to mount your S3 bucket as a directory in your EC2 server.

I used this tutorial and I was able to successfully mount my S3 to my EC2 running on Ubuntu 14.04.

WARNING You are billed in S3 based on the number of request you make. I did not do any further research, but my S3 usage spiked when I did this. I believe that the connections maintained by the mounted S3 bucket was counted as a GET/PUT/LIST request.

Another Alternative is to use Amazon's Elastic File System. This is like S3 but a file system.

3
  • s3fs had some performance issues for me, but it is only way how to achieve it. Jun 1, 2016 at 12:45
  • Some of the default settings of s3fs need to be changed, but when properly configured it does work pretty well... But it is important to remember that, as mentioned, S3 is not a filesystem. Also, you can get pretty close to "infinite" with the new EBS Cold Storage volume class. It has its own design limitations, but it it's a real block device where you can put a real filesystem, and $25/mo per provisioned terabyte isn't bad. Jun 2, 2016 at 1:20
  • I believe that the connections maintained by the mounted S3 bucket was counted as a GET/PUT/LIST request. There are no connections maintained. S3 access is only/always via REST requests and s3fs uses libcurl. But for each ls -l, at least one LIST request is followed by a HEAD request for each file in the directory, because the S3 object metadata is where the unix owner, group, and file mode are stored. So, uses, this can add up. Jun 2, 2016 at 1:25
1

What you're looking for is EFS(elastic file system), but that hasn't been released yet. What you could try is this: https://forums.aws.amazon.com/thread.jspa?messageID=475840 but I have never tried this myself.

3
  • ~EFS is already available.~
    – julio
    Jun 1, 2016 at 11:10
  • No, it's not. Only as preview in only 2 regios and you need to request access.
    – MaartenS
    Jun 1, 2016 at 11:13
  • MaartenS is correct. Its NOT yet available.
    – julio
    Jun 1, 2016 at 11:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .