Questions tagged [s3fs]

s3fs is FUSE-based filesystem that stores files in AWS S3.

Filter by
Sorted by
Tagged with
2
votes
1answer
77 views

Amazon s3 bucket mount point getting unmounted automatically

I use s3fs on aws ubuntu instance to mount s3 bucket. Following is the command I used to mount the bucket. sudo fusermount -u /s3/project1 sudo s3fs -o allow_other project1 /s3/project1 The ...
2
votes
1answer
96 views

Use S3FS Mount As HTTP Server Document Root

My proposed setup: An EC2 server running Apache or nginx. An S3 Bucket in the same region/availability zone mounted with S3FS. The Apache or nginx document root set to the S3FS mount. My question: ...
0
votes
0answers
16 views

s3fs POSIX permissions on Google Storage

Does s3fs support POSIX permissions uploaded to Google Storage with gsutil cp -P? Or do you need to mount the bucket with s3fs and copy in the files to preserve file permissions?
0
votes
1answer
253 views

s3fs does not pharse NO_PROXY / no_proxy env variables

I am running s3fs on a node in corporate network which is behind proxy. I am trying to mount S3 bucket on another node next to it. Both nodes are in 10.88.97.65/27 network and should be able to ...
2
votes
2answers
2k views

Can not unmount an S3 directory mounted with s3fs-fuse

I have an s3fs-fuse "device" mounted at /mnt/s3. I want to unmount it so that I can remount it with the mp_umask flag so that chroot will work properly. The command should be: fusermount -u /mnt/s3 ...
3
votes
1answer
852 views

Automatically mounting S3 bucket using s3fs on Amazon CentOS

I have tried all the answers provided in similar questions but none is helpful. I installed S3 Fuse so that I can mount S3 bucket. After the installation, I performed the following the steps: I ...
2
votes
1answer
569 views

How to use SAMBA on an Amazon S3 mounted directory on Ubuntu VM

I had successfully configured a SAMBA share on /home/myusername on an Ubuntu 17.10 VM. I then used s3fs-fuse to mount an S3 bucket on that exact directory, i.e. /home/myusername. It worked too, any ...
1
vote
1answer
138 views

S3 Bucket for hosting media, s3fs mount catch-22

I'm attempting to use an S3 bucket to host media files (images, video and static files) for a small web application and came across a strange catch-22... According to the S3 documentation, to serve a ...
1
vote
0answers
323 views

sharing a S3 storage mount via samba: Permission denied

At work we've recently deployed several S3 storage servers. However, not all of our customers have the ability to work with S3 storage due to old hardware/software. We tried to work around the ...
1
vote
0answers
72 views

Subdomain pointing to deleted S3 bucket [duplicate]

I am stuck in my present (AP-SOUTH) and deleted (US-EAST) S3 bucket. I had mistakenly created a bucket in US-EAST (cdn.example.com.s3-website-us-east-1.amazonaws.com) and mapped my subdomain - cdn....
1
vote
1answer
4k views

Automatically mount bucket with s3fs on boot

I use an Amazon S3 bucket to deliver some of my server's content. I was able to mount it successfully, and grant Apache rights over it, but can't get it mounted properly at reboot. I updated my /etc/...
1
vote
0answers
316 views

rpc.mountd[pid]: /xxx and /yyy have same filehandle for host, using first

I have an S3 bucket mounted using s3fs-fuse at /yyy and a normal drive mounted at /xxx. I am exporting both with NFS however when a client attempts mount, I see the following error: rpc.mountd[pid]: /...
-1
votes
2answers
3k views

AWS S3 no space left on device

I am mounting a S3 bucket on a amazon EC2 instance using S3FS. Things have been working great for now over 18 months. Today I've tried to download a 5GB file using wget directly on the S3 bucket (...
1
vote
1answer
569 views

Best manner to access files in S3 Bucket from EC2 instance

I have tried several packages (goofys, s3fs, fuse) and have not been able to find a solution to my problem. I have an S3 Bucket with several thousand files currently in it and will continue to grow. ...
1
vote
1answer
2k views

Custom file/folder permissions with S3FS?

I have setup an SFTP server using OpenSSH. The home directory for users is /sftp/%user. I'm mounting an S3 bucket at /sftp using S3FS. The problem is that S3FS cascades user permissions down through ...
1
vote
0answers
427 views

Root user permission issues with SFTP and S3FS on Amazon EC2

I have setup an SFTP server using OpenSSH. The home directory for users is /sftp/%user. I'm mounting an S3 bucket at /sftp using S3FS. The problem is that S3FS cascades user permissions down through ...
4
votes
0answers
1k views

Configure SFTP with OpenSSH and an AWS S3 Bucket mounted via S3FS on Amazon EC2

How do I allow multiple SFTP Users with S3FS and OpenSSH? Everything works, except SFTP Users don't have permission to write to their Chrooted Home Directory: remote open("/some_file"): Permission ...
0
votes
0answers
570 views

S3FS and Fuse, scale out or scale up

I have an infra where I have multiple machines (eg FTP servers, get/put and Samba/NFS servers) that need to share out the same (HA) storage. So I figured to use s3fs and fuse. I got this all setup and ...
0
votes
1answer
153 views

Automate s3fs-fuse install

I'm trying to write script to install s3fs-fuse, but i don't know how to change directory in a bash script. These are the script I want to automate git clone https://github.com/s3fs-fuse/s3fs-fuse....
0
votes
1answer
546 views

Is S3 a proper choice to keep live linux user home directories?

We are building an RDP cluster using Amazon EC2/S3. Users' home directories are on S3 (single bucket) and all EC2 nodes mount the same S3 bucket using S3FS. Each EC2 node uses the appropriate home ...
5
votes
3answers
3k views

Replicated shared filesystem

I'm looking into setting up a shared filesystem/file server on AWS (EC2) infrastructure that offers replication and fairly painless failover. This filesystem would host potentially millions of files ...
2
votes
1answer
4k views

s3fs “could not determine how to establish security credentials”

I'm trying to use s3fs to mount an S3 bucket on to a standard AWS Amazon Linux AMI (with all the necessary dependencies installed). However when following this tutorial when I run: s3fs mybucketname -...
0
votes
1answer
529 views

Does linux count s3fs mounts as local disk usage?

I have an s3 bucket mounted on an EC2 instance whose main disk is an EBS volume. Whenever I run something like du -h /path/to/mount it tells me the size of the folders within those. I'm wondering ...
0
votes
1answer
1k views

AWS: Best solution for syncing files across autoscaled EC2 instances?

I'm working on setting up autoscaling on EC2; Instances will be running a web app with big traffic spikes. The first solution I tried for syncing files across instances (since the web app will write ...
1
vote
0answers
469 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
5
votes
1answer
2k views

Linux server sync to an Amazon S3 bucket

I am looking for a stable solution to replace a classic server backup to another server using rsync. I have to sync a whole filesystem (more than 1Tb) to Amazon S3. Where am I? Solution 1: I mapped ...
1
vote
1answer
714 views

Fuse module for s3fs

I am trying to install the s3fs software. I however keep coming upon this error: $ sudo /usr/bin/s3fs bucketname /mnt fuse: device not found, try 'modprobe fuse' first $ modprobe fuse FATAL: Module ...
0
votes
1answer
171 views

autoscale static content on ec2 using local cache

Hello Serverfault members. I want to achieve the following setup on EC2: A load balancer starts and stops instances which only serve static content. (a javascript based application) I do not want to ...
0
votes
1answer
147 views

Subsequent runs of rsync locally don't reduce data transferred

I have an EC2 instance with data I want to sync to a mounted, but remote, volume, as a backup. rsync seems like the way to go with this, so as a test I took my test file (a Postgres pg_dump file) and ...
0
votes
1answer
438 views

configure s3fs mount for write-only

I am using s3fs to mount a S3 buckets on an Ubunutu server to store my daily backups there. Considering security this would allow a potential hacker to delete all my backups from the mounted bucket. ...
5
votes
4answers
9k views

How to make s3fs work with IAM roles?

We are using IAM roles to maintain the machines. We now planning to mount the s3 to our ec2 instances and do the processing as per our need. We been using s3fs tool for mounting. But that doesn't ...
1
vote
4answers
622 views

Shared encrypted filesystem for ec2 instances

Me and my company intend to start using ec2 for distributing some cpu-intensive pieces code, involving a few 10s of gb of data, however I have some concern about the protection of the intellectual ...
1
vote
1answer
905 views

Getting errors running mount on Amazon Linux after installing s3fs (`MOUNT_2.22' not found required by mount)

Followed these instructions to install s3fs on Amazon Linux. It worked fine on one of my systems, however on a different system (following the same instructions) I am getting the following errors ...
2
votes
0answers
438 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
9
votes
7answers
18k views

S3fs performance improvements or alternative?

I've been trying to use s3fs on an EC2 server but it's really slow. I spent the last 5 hours uploading 100MB of small files. Is there anything I can do to improve the performance or is it something ...
2
votes
2answers
4k views

s3fs changing s3 permissions?

My developer believes that s3fs is changing my bucket's permissions. Is this possible? I want my bucket to be public, but it keeps reverting back to private. Here's my fstab. s3fs#production /mnt/...
1
vote
3answers
2k views

How to list s3fs mounts?

How can I list the s3fs mounts that exist on an ubuntu system? I'd like to know to which bucket each mount is mapped. Specifically, I have a specific mount (e.g. ~/s3/mymount), and would like to know ...
9
votes
2answers
4k views

EC2 - Shared Storage - S3FS or EBS?

Building my web service onto EC2 right now and have a single instance behind a load balancer. I will of course cater for multiple instances. My initial idea was to run all the instances a dumb slaves,...
1
vote
1answer
2k views

Mounting an S3 bucket onto a AWS Ubuntu instance issues

i'm using s3fuse to mount an s3 bucket onto ubuntu, so that i can take advantage of the storage space. here are some environment info: System Info: Linux ip-xxx-xxx-xxx-xxx 2.6.38-8-virtual #42-...
2
votes
1answer
2k views

Writing to an s3 mounted directory

hopefully someone can advise. I've recently mounted an s3 bucket to a directory /s3 on my ubuntu amazon ec2 instance -following: http://code.google.com/p/s3fs/wiki/FuseOverAmazon. This was set up in ...
5
votes
2answers
962 views

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which ...
0
votes
1answer
2k views

s3fs and fuse: failed to open /dev/fuse: Permission denied

Just installed Debian 6 box. I wanted to mount my S3 bucket for some backups. So I followed instructions for http://code.google.com/p/s3fs/ . 1) I installed all perquisites asked in installation ...
0
votes
3answers
950 views

Bacula over s3fs

We are constantly running out of space on our backup server so I want to move the backups to S3. Does it make sense to run Bacula on top of s3fs? Does anybody have experience with that? Is it reliable?...
3
votes
1answer
3k views

Why is mount -a not mounting fuse drive properly when executed remotely (via Fabric)?

This is a weird bug and I'm not sure where it's coming from. Here's a quick run down of what I'm doing. I'm trying to mount a FUSE drive to an Amazon EC2 instance running Ubuntu 10.10 using s3fs (...
3
votes
1answer
3k views

s3fs or EBS through nfs? [closed]

Which one is better? I need to store a lot of static files ... and I wonder which one is better, cheaper and faster.
3
votes
3answers
3k views

s3fs: how to force remount on errors?

I use s3fs 1.33 on Ubuntu 9.10. Regularily it gives me errors like this: rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32) rsync: close failed on "/mnt/s3/...
7
votes
1answer
13k views

How to force s3fs mount on boot

I'm using s3fs 1.33 on Ubuntu 9.10 x86. I mount it manually with this command: sudo /usr/bin/s3fs mybucket /mnt/s3/mybucket -ouse_cache=/tmp How can I force the mount on boot? I guess that I have ...
4
votes
4answers
4k views

Best remote mounted filesystem based on S3 [closed]

I need to know what the best (read: highly performant and fault tolerant) filesystem solution based on Amazon's S3 is. I have looked a s3fs, but I notice that it has not been worked on for a while now,...
4
votes
3answers
3k views

s3fs Input/output error

I'm trying to set up a backup system with s3fs and the Amazon S3 service. I followed this this guide and this guide. Tailing /var/log/messages I get: Aug 28 13:37:46 server s3fs:###response=403 I ...