Questions tagged [s3fs]

s3fs is FUSE-based filesystem that stores files in AWS S3.

Filter by
Sorted by
Tagged with
0 votes
1 answer
60 views

mount a readonly public s3 bucket with s3fs

We have minio server with a public readonly bucket. I want to mount it using s3fs but cannot figure out how to mount it. s3fs always wants credentials from me.
  • 105
0 votes
1 answer
166 views

Bandwidth consumption of Amazon S3 bucket mounted with S3fs on a VPS

I have : VPS server on Vultr with web server and Wordpress installed. Uploads folder is mounted on Amazon S3 bucket using s3fs. If a user downloads a 1GB file from my website will the traffic be ...
0 votes
1 answer
425 views

Cannot mount AWS S3 bucket/path using s3fs on CentOS 7

From a CentOS 7 virtual machine running on Azure, I am unable to mount an AWS S3 bucket using s3fs. The path part of the bucket location is removed before s3fs checks for bucket's existence: s3fs ...
0 votes
2 answers
462 views

Unable to mount a folder in Ubuntu to as AWS S3 Bucket

I am attempting to mount a folder on a fresh AWS EC2 instance running Ubuntu 18.04, to AWS S3. I've been following instructions I found at https://cloud.netapp.com/blog/amazon-s3-as-a-file-system and ...
0 votes
1 answer
376 views

Experience with using Openstack Swift with s3fs multi user access?

Does anyone have experience with using Openstack Swift object storage with s3fs and having multiple write mounts accessing the same file(s) at the same time? The man page for s3, for example: https://...
  • 1
1 vote
0 answers
616 views

S3FS mount does not work as non-root user

I'm trying to mount an Exoscale bucket to an Exoscale VM running Ubuntu 20.04 using s3fs. After reading the s3fs README and a few online tutorials here what I have done. Please, bear in mind, I'm ...
  • 131
0 votes
1 answer
1k views

chmod/chown/mkdir: Operation not permitted as root

I get Operation not permitted for root user on one server but not on the other "identical" server. Running on Amazon Linux 1. Server 1: [root@preprod-1 ]# chown root:root /s3mnt/outliers/ ...
1 vote
1 answer
1k views

S3FS Mount does not seem to work: Am I doing something wrong?

So I've just a brand new VPS which I'll be using for nextcloud. I followed this guide. I did (on a brand new machine): Apt install s3fs Create a /etc/passwd-s3fs file with key:secret key Chmoded it ...
0 votes
2 answers
1k views

Mount AWS S3 on boot while behind proxy using s3fs

In order to get a mount to work I have to first set the http_proxy environment variable before I run s3fs. export https_proxy=<proxy address> s3fs -d <bucket> mount-point -o url=https://...
  • 157
1 vote
3 answers
728 views

mysqldump using a lot of space

I have a server and the / partition is 20GB in size. Databases are stored in /mnt/mysql-data partition is 500GB in size. Now here's the problem. Whenever I run mysqldump it fills up / partition to ...
5 votes
0 answers
752 views

How does S3FS (or any other S3 FUSE filsystem) compare to AWS FSx for Lustre + S3

I remember trying s3fs a year back, trying to use some S3 bucket as a FUSE filesystem. I remember it being quite laggy, especially when coupled with git operations on it (an oblivious system architect ...
  • 225
2 votes
1 answer
2k views

Amazon s3 bucket mount point getting unmounted automatically

I use s3fs on aws ubuntu instance to mount s3 bucket. Following is the command I used to mount the bucket. sudo fusermount -u /s3/project1 sudo s3fs -o allow_other project1 /s3/project1 The ...
  • 141
2 votes
1 answer
466 views

Use S3FS Mount As HTTP Server Document Root

My proposed setup: An EC2 server running Apache or nginx. An S3 Bucket in the same region/availability zone mounted with S3FS. The Apache or nginx document root set to the S3FS mount. My question: ...
  • 384
3 votes
1 answer
692 views

How to share a EFS/s3 with code in a auto scaling group?

the main problem is: Infrastructure: Autoscaling (min 1 - max 3) RDS ELB elasticache (redis) elasticsearch I want to share a volume (EFS / S3) with the code of my application; The size of it is ...
0 votes
1 answer
666 views

s3fs does not parse NO_PROXY / no_proxy env variables

I am running s3fs on a node in corporate network which is behind proxy. I am trying to mount S3 bucket on another node next to it. Both nodes are in 10.88.97.65/27 network and should be able to ...
6 votes
2 answers
17k views

Can not unmount an S3 directory mounted with s3fs-fuse

I have an s3fs-fuse "device" mounted at /mnt/s3. I want to unmount it so that I can remount it with the mp_umask flag so that chroot will work properly. The command should be: fusermount -u /mnt/s3 ...
3 votes
1 answer
4k views

Automatically mounting S3 bucket using s3fs on Amazon CentOS

I have tried all the answers provided in similar questions but none is helpful. I installed S3 Fuse so that I can mount S3 bucket. After the installation, I performed the following the steps: I ...
  • 33
3 votes
1 answer
2k views

How to use SAMBA on an Amazon S3 mounted directory on Ubuntu VM

I had successfully configured a SAMBA share on /home/myusername on an Ubuntu 17.10 VM. I then used s3fs-fuse to mount an S3 bucket on that exact directory, i.e. /home/myusername. It worked too, any ...
1 vote
1 answer
435 views

S3 Bucket for hosting media, s3fs mount catch-22

I'm attempting to use an S3 bucket to host media files (images, video and static files) for a small web application and came across a strange catch-22... According to the S3 documentation, to serve a ...
1 vote
0 answers
736 views

sharing a S3 storage mount via samba: Permission denied

At work we've recently deployed several S3 storage servers. However, not all of our customers have the ability to work with S3 storage due to old hardware/software. We tried to work around the ...
  • 1,297
1 vote
0 answers
107 views

Subdomain pointing to deleted S3 bucket [duplicate]

I am stuck in my present (AP-SOUTH) and deleted (US-EAST) S3 bucket. I had mistakenly created a bucket in US-EAST (cdn.example.com.s3-website-us-east-1.amazonaws.com) and mapped my subdomain - cdn....
  • 11
5 votes
2 answers
15k views

Automatically mount bucket with s3fs on boot

I use an Amazon S3 bucket to deliver some of my server's content. I was able to mount it successfully, and grant Apache rights over it, but can't get it mounted properly at reboot. I updated my /etc/...
  • 294
2 votes
0 answers
657 views

rpc.mountd[pid]: /xxx and /yyy have same filehandle for host, using first

I have an S3 bucket mounted using s3fs-fuse at /yyy and a normal drive mounted at /xxx. I am exporting both with NFS however when a client attempts mount, I see the following error: rpc.mountd[pid]: /...
1 vote
2 answers
8k views

AWS S3 no space left on device

I am mounting a S3 bucket on a amazon EC2 instance using S3FS. Things have been working great for now over 18 months. Today I've tried to download a 5GB file using wget directly on the S3 bucket (...
1 vote
1 answer
705 views

Best manner to access files in S3 Bucket from EC2 instance

I have tried several packages (goofys, s3fs, fuse) and have not been able to find a solution to my problem. I have an S3 Bucket with several thousand files currently in it and will continue to grow. ...
3 votes
1 answer
4k views

Custom file/folder permissions with S3FS?

I have setup an SFTP server using OpenSSH. The home directory for users is /sftp/%user. I'm mounting an S3 bucket at /sftp using S3FS. The problem is that S3FS cascades user permissions down through ...
1 vote
0 answers
620 views

Root user permission issues with SFTP and S3FS on Amazon EC2

I have setup an SFTP server using OpenSSH. The home directory for users is /sftp/%user. I'm mounting an S3 bucket at /sftp using S3FS. The problem is that S3FS cascades user permissions down through ...
4 votes
1 answer
2k views

Configure SFTP with OpenSSH and an AWS S3 Bucket mounted via S3FS on Amazon EC2

How do I allow multiple SFTP Users with S3FS and OpenSSH? Everything works, except SFTP Users don't have permission to write to their Chrooted Home Directory: remote open("/some_file"): Permission ...
10 votes
2 answers
8k views

How to load balance sftp instances on AWS

I like to know is it possible to load balance sftp servers in AWS. I have 2 servers, and each of my servers are using s3fs-fuse to mount the same S3 bucket onto a mount point. Both of my ec2 ...
  • 201
0 votes
0 answers
811 views

S3FS and Fuse, scale out or scale up

I have an infra where I have multiple machines (eg FTP servers, get/put and Samba/NFS servers) that need to share out the same (HA) storage. So I figured to use s3fs and fuse. I got this all setup and ...
0 votes
1 answer
197 views

Automate s3fs-fuse install

I'm trying to write script to install s3fs-fuse, but i don't know how to change directory in a bash script. These are the script I want to automate git clone https://github.com/s3fs-fuse/s3fs-fuse....
  • 117
0 votes
1 answer
957 views

Is S3 a proper choice to keep live linux user home directories?

We are building an RDP cluster using Amazon EC2/S3. Users' home directories are on S3 (single bucket) and all EC2 nodes mount the same S3 bucket using S3FS. Each EC2 node uses the appropriate home ...
5 votes
3 answers
4k views

Replicated shared filesystem

I'm looking into setting up a shared filesystem/file server on AWS (EC2) infrastructure that offers replication and fairly painless failover. This filesystem would host potentially millions of files ...
  • 471
4 votes
3 answers
10k views

s3fs "could not determine how to establish security credentials"

I'm trying to use s3fs to mount an S3 bucket on to a standard AWS Amazon Linux AMI (with all the necessary dependencies installed). However when following this tutorial when I run: s3fs mybucketname -...
0 votes
1 answer
1k views

Does linux count s3fs mounts as local disk usage?

I have an s3 bucket mounted on an EC2 instance whose main disk is an EBS volume. Whenever I run something like du -h /path/to/mount it tells me the size of the folders within those. I'm wondering ...
  • 201
0 votes
1 answer
1k views

AWS: Best solution for syncing files across autoscaled EC2 instances?

I'm working on setting up autoscaling on EC2; Instances will be running a web app with big traffic spikes. The first solution I tried for syncing files across instances (since the web app will write ...
  • 501
1 vote
0 answers
540 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
  • 193
5 votes
1 answer
2k views

Linux server sync to an Amazon S3 bucket

I am looking for a stable solution to replace a classic server backup to another server using rsync. I have to sync a whole filesystem (more than 1Tb) to Amazon S3. Where am I? Solution 1: I mapped ...
  • 141
1 vote
1 answer
1k views

Fuse module for s3fs

I am trying to install the s3fs software. I however keep coming upon this error: $ sudo /usr/bin/s3fs bucketname /mnt fuse: device not found, try 'modprobe fuse' first $ modprobe fuse FATAL: Module ...
  • 193
0 votes
1 answer
193 views

autoscale static content on ec2 using local cache

Hello Serverfault members. I want to achieve the following setup on EC2: A load balancer starts and stops instances which only serve static content. (a javascript based application) I do not want to ...
  • 113
1 vote
1 answer
387 views

What's the best way to enforce a maximum size on a file directory?

I'd like to create directory with a maximum size of, let's say 500MB. The directory may contain any number and depth of subdirectories. When the maximum size limit of the directory is surpassed, I ...
  • 501
0 votes
1 answer
160 views

Subsequent runs of rsync locally don't reduce data transferred

I have an EC2 instance with data I want to sync to a mounted, but remote, volume, as a backup. rsync seems like the way to go with this, so as a test I took my test file (a Postgres pg_dump file) and ...
  • 115
0 votes
1 answer
569 views

configure s3fs mount for write-only

I am using s3fs to mount a S3 buckets on an Ubunutu server to store my daily backups there. Considering security this would allow a potential hacker to delete all my backups from the mounted bucket. ...
15 votes
5 answers
21k views

How to make s3fs work with IAM roles?

We are using IAM roles to maintain the machines. We now planning to mount the s3 to our ec2 instances and do the processing as per our need. We been using s3fs tool for mounting. But that doesn't ...
  • 277
2 votes
2 answers
5k views

Can a S3 mount be used as the document root for Apache?

Has anyone been successful in having their DocumentRoot reside on an S3 mount (using s3fs)? I currently have a mounted bucket at /mnt/s3. I can read and write files to it no problem. In my httpd....
  • 21
1 vote
4 answers
654 views

Shared encrypted filesystem for ec2 instances

Me and my company intend to start using ec2 for distributing some cpu-intensive pieces code, involving a few 10s of gb of data, however I have some concern about the protection of the intellectual ...
  • 111
1 vote
1 answer
1k views

Getting errors running mount on Amazon Linux after installing s3fs (`MOUNT_2.22' not found required by mount)

Followed these instructions to install s3fs on Amazon Linux. It worked fine on one of my systems, however on a different system (following the same instructions) I am getting the following errors ...
  • 1,265
2 votes
0 answers
524 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
  • 131
18 votes
9 answers
34k views

S3fs performance improvements or alternative?

I've been trying to use s3fs on an EC2 server but it's really slow. I spent the last 5 hours uploading 100MB of small files. Is there anything I can do to improve the performance or is it something ...
  • 385
2 votes
2 answers
5k views

s3fs changing s3 permissions?

My developer believes that s3fs is changing my bucket's permissions. Is this possible? I want my bucket to be public, but it keeps reverting back to private. Here's my fstab. s3fs#production /mnt/...
  • 169