I've been trying to use s3fs on an EC2 server but it's really slow. I spent the last 5 hours uploading 100MB of small files.

Is there anything I can do to improve the performance or is it something related to S3fs? If not, what's an alternative I could use?

  • define small files, 1K? 500bytes? 1MB?
    – thinice
    Jun 6, 2012 at 14:21
  • around ~200K...
    – MB.
    Jun 6, 2012 at 14:22
  • Depending on your storage needs, you may want to just use EBS, unless you need access from other systems simultaneously, in which case setting up a virtual nas with EBS may still be a better option.
    – Tracker1
    Dec 20, 2014 at 22:45

9 Answers 9


S3FS may not be the best choice for a large amount of smaller files. The overhead with S3FS is pretty high as well. I'd suggest using something like S3Curl

You can even get parallel transfers going. Just remember it will never be fast like EBS / local storage.

If you need to have it as a 'mountable' storage, the only alternatives to S3FS that I know of is S3Backer or s3ql

  • 1
    I'm looking for something that I can mount as a filesystem.
    – MB.
    Jun 6, 2012 at 14:27
  • Any thoughts on s3ql?
    – ewwhite
    Jun 6, 2012 at 14:29
  • I'll try S3Backer and S3Sql. Thanks.
    – MB.
    Jun 6, 2012 at 14:32
  • 5 hours is really slow for ~500, 200KB files even with S3FS. Something else might be factoring in, I'm not aware of any S3FS tunable that would help
    – thinice
    Jun 6, 2012 at 14:32

I just released v0.0.1 of https://github.com/kahing/goofys which was partly motivated by performance problems in s3fs. File creation speedup is 3-6x and time to first byte is 58x. Feedback welcome!

  • 3
    As of 2019 - Goofys is the recommended choice. Riotfs hasn't been updated in a while. Sep 30, 2019 at 1:38
  • 2
    2021 checking in - Goofys is incredible. According to my droplet graphs I've gone from 20% CPU to 1% CPU, load is now negligible, mem usage is flat at 20% (instead of spiking to 80%), and bandwidth is down from a sustained 1Gbps to under 500 kbps. I don't understand it, but the same workload is running and average operation times dropped from 6.5 seconds to 1.8. And the worst case is 10 vs 110. These are small random reads in very large files. May 12, 2021 at 17:39

I just benchmarked riofs compared to s3fs. My test case was a relatively simple bash script that ran pngquant on each .png it found. On a testbucket with ~70 images of which ~20 where png (spread over a lot of subdirectories, which prob slows things down) the results where:

s3fs: 3m54
riofs: 15.9s

So for this testcase riofs is ~15x faster! Setup was also very straightforward, although the docs are somewhat terse.

Regarding the fact that the script still took 15.9s: it's not very efficient, also actually running pngquant on png's is a cpu intensive process.

  • Unfortunately riofs fails with AWS message: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
    – lrkwz
    Jan 23, 2017 at 16:56


Goofys allows you to mount an S3 bucket as a filey system.

It's a Filey System instead of a File System because goofys strives for performance first and POSIX second. Particularly things that are difficult to support on S3 or would translate into more than one round-trip would either fail (random writes) or faked (no per-file permission). Goofys does not have an on disk data cache (checkout catfs), and consistency model is close-to-open.


I used [1] and it's very good. It's lot of commands and makes accessing aws and s3 a lot easier.

  1. http://timkay.com/aws/
  • 1
    Welcome to Server Fault! Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.
    – Scott Pack
    Oct 1, 2012 at 2:27

Just adding riofs to the mix. I could not see any benchmark for now but there is an issue/request that anybody can do.


I compared s3fs with riofs, and almost switched because of noticeably better performance. Problem is riofs is still not updated to v4 signatures, which means it won't work with Frankfurt or China s3 buckets. According to the owner, an update is on the way. If you don't use Frankfurt or China, riofs is a hands down better option.

  • Does not work from eu-central-1 either
    – lrkwz
    Jan 23, 2017 at 16:57

This is kind old but may help who gets here.

I'm using aws cli and its working very good (3 Mb/s)

pip install awscli --upgrade --user
nano ~/.bash_profile
export PATH="~/.local/bin:$PATH"
aws s3 sync /sourcedir/ s3://bucketname/destdir/

Used rclone mount and is far much faster than s3fs

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.