This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

3
votes
0answers
163 views

Set up a different index document per directory with S3 static website hosting

I am using S3 static website hosting, and trying to get the following redirects to work: example.com --> example.com/index.html example.com/foo --> example.com/foo/foo.html example.com/bar --&...
3
votes
0answers
73 views

S3 + Cloudflare vs S3 + 2 Web Server + Cloudflate

Currently, I run two websites and each site own it own big server. Currently, all of those two sites has roughly 300GB of images. Its grow rate is 15GB of images per month. I know that eventually, one ...
3
votes
0answers
127 views

Solution for file data backup, versioning and replication

We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
3
votes
0answers
458 views

Strict-Transport-Security on CloudFront with S3 origin?

My company's site has a static homepage for speed and cost reasons. We use S3 as the origin for CloudFront. Now, we would like to declare Strict-Transport-Security for the entire domain, but S3 seems ...
2
votes
0answers
753 views

AWS CLI throws “Unable to locate credentials”, the second time it's run

I'm trying to download some files from S3 to an EC2 instance using the AWS CLI (1.8.7) on startup using a User-Data Script. This instance has a IAM Role & Instance Profile with the appropriate ...
2
votes
0answers
997 views

Nginx as s3 proxy with private buckets

I'm trying to set up Nginx as a proxy for a static html/js webapp hosted on S3. I read a lots of tutorials and posts and I could make it work if my bucket is set as public. The reason I need Nginx as ...
2
votes
0answers
2k views

How can I move s3 objects >5gb from one bucket to another?

Objects larger than 5GB are supported with multi-part uploading. We have 5gb+ files in one bucket already that I'd like to move to another, under the same AWS account. When I issue the command using ...
2
votes
0answers
243 views

Gluster with s3ql mounted brick?

I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount. Having just the plain directory as a unique ...
2
votes
0answers
540 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
1
vote
0answers
36 views

S3/Cloudfront CORS working on some files only

I'm really struggling to get Cloudfront and S3 to add Access-Control-Allow-Origin: * to the headers of video files stored on S3 (for inline video on iPhones - seems to be working everywhere else ...
1
vote
0answers
28 views

fast backup script in linux

I have a server with more applications and do normal shell command to take backups. it took couple of days to complete. Tried to install duplicity. but its told some packages are not found. its Debian ...
1
vote
0answers
17 views

Enabling S3 bucket as website using cli

I figured out how to create an S3 bucket using aws command line. Is it possible to enable the s3 bucket I created as website using aws cli? The reason I'm doing this is because we're automating ...
1
vote
0answers
32 views

Is there a way to get the size of Amzon S3 bucket from S3 console?

I wonder if there is some way to get the size of Amazon S3 bucket directly from console https://console.aws.amazon.com/s3/ Clicking properties returns great deal of information, but not the size of ...
1
vote
0answers
55 views

Bucket policy that respects pre-signed URLs OR IP Address deny?

I would like to be able to restrict access to files in a S3 bucket in multiple ways. This is due to the fact that the files stored can be accessed in different manners. We do this because we have TBs ...
1
vote
0answers
169 views

aws s3 sync: how to exclude proc, sys directories

I can't seem to get the aws sync s3 command to exclude directories. I am backing up a mounted filesystem from OpenVZ which is located at /vz/root/100. My command is: aws s3 sync /vz/root/100 s3://[...
1
vote
0answers
203 views

CloudFront + S3 Access Denied Using Signed Cookies

I'm trying to integrate signed cookies for a CloudFront distribution that's fed off of an S3 bucket but getting access denied error messages: <Error> <Code>AccessDenied</Code> ...
1
vote
0answers
766 views

s3cmd sync hangs forever. Never finishes

I am trying to backup a directory of around 350GB in size to S3. The command I run is s3cmd --progress -v -H sync /localdir s3://mybucket/ However, the process never moves past INFO: Compiling ...
1
vote
0answers
58 views

svnadmin dump returns error insufficient disk space

I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked. I have Apache Subversion hosted on an ...
1
vote
0answers
234 views

Duplicity -> S3 backup failing with “[Errno 105] No buffer space available”

I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
1
vote
0answers
641 views

Amazon S3: allow user access to subfolder only on any bucket

I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket. This is my policy so far (reading works, but it ...
1
vote
0answers
139 views

Handling multiple server blocks in nginx configuration

I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com. Eg:If ...
1
vote
0answers
297 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
1
vote
0answers
351 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
0
votes
0answers
20 views

Recover code from encrypted partition Amazon server

We are thinking of a way to be able to recover code that a former developer left on our server. We have full access to the Amazon account, but not SSH access and the partition has a password ...
0
votes
0answers
10 views

Is there an S3 storage backend for varnish cache?

I want to run varnish cache in a docker container on AWS ECS. If the container dies and a new one takes its place the new one should have easy immediate access to the the cached data so it can ...
0
votes
0answers
16 views

Access denied for sub-paths on CloudFront

I have a distribution on Cloudfront that use a S3 bucket. The first page works but not he sub-paths. For example, when I go to 'www.example.com' it works but not 'www.example.com/admin'. I always ...
0
votes
0answers
20 views

list s3 object permissions via SDK / CLI / API?

I want to check which objects in the bucket are public available. I can run $s3->listObjectsV2 successfully, but I do not see any permissions here at all. What am I doing wrong? Also, I'd like to ...
0
votes
0answers
44 views

Permission issue in Riofs and Fuse with non root users

I am using fuse with riofs to mount local directory to s3 bucket. Below is the command I am running on the server. sudo /usr/local/bin/riofs -v -c /root/.config/riofs/riofs.conf.xml -l /var/log/...
0
votes
0answers
9 views

AmazonS3 Exception in Windows Vista

I got an app that access and query data from AWS s3, it works fine in pcs with Win7, 8.1 and 10 (with NetFramework 4 or 4.5) But it fails when i run the same app in a PC with Windows Vista, it throws ...
0
votes
0answers
32 views

Is not authorized to perform an operation while I have the full access of an aws service

I have created a simple nodejs script to handle the files in S3, and I want it to be executed on lambda. My admin has added me in IAM with full access to lambda and S3. However, when I select the "S3 ...
0
votes
0answers
31 views

Does CloudFront support S3 signature version 4 for KMS encrypted objects?

I'm using Cloudfront with an S3 origin that is using KMS to encrypt objects. I'm getting the following error when sending a GET request for an object in the S3 bucket. Requests specifying Server ...
0
votes
0answers
38 views

Redirect one root domain to another all on AWS Route 53 / S3

Folks. I have been trying to configure a new domain I just bought to redirect to one I have successfully running on AWS Route 53, but cannot make this work. I am somewhat of a novice and have tried ...
0
votes
0answers
56 views

Reverse Proxy on cloudfront

I have an angular site / web application (Lets call this www.mysite.com) hosted on Amazon S3 as a static website. This s3 bucket serves as the origin for a cloudfront distribution and it works ...
0
votes
0answers
18 views

AWS S3 “service user” for non-AWS server?

I'm setting up a server outside of AWS, but I'd like it to read and write from one of my AWS S3 buckets. I know that within AWS, best practice is to assign a role with S3 read/write permission to the ...
0
votes
0answers
52 views

AWS CLI copy life-cycle configuration between buckets

I am trying to read-modify-write an existing life-cycle configuration. So the simpler thing I am staring out with is copying a life-cycle configuration between buckets. From reading through the ...
0
votes
0answers
14 views

Google Storage Sync from Amazon S3 is not running

I setup a new Google Storage bucket, then added the username and password from the S3 permissions, including the correct policy as instructed on Amazon S3 and then scheduled the sync for 3 hours in ...
0
votes
0answers
36 views

Syncing from a NAS drive to AWS S3 with s3cmd

I'm trying to use s3cmd to sync data from a NAS drive connected to a web server node, to AWS S3. I'm using this command: s3cmd sync ./logs/ s3://bucket/logs/ What's happening is there is a massive ...
0
votes
0answers
163 views

Nginx Proxy pass with aws s3 gives 403

I have an web app that uploads images in server and copies them to s3 bucket using awscli. Nginx on server retrieves images from s3 through reverse proxy. When I try to open image using url, it gives ...
0
votes
0answers
50 views

Read Files from S3FS mount with PHP

We have a S3 bucket we use for video uploading. I also have a script for accessing the Youtube API so I can pull the upload files from a server rather than via the Web interface. This used to work ...
0
votes
0answers
37 views

When s3cmd checks a bucket with thousands of files to sync it, does this count as one list request or thousands?

Trying to figure out pricing here so I don't get overcharged. I daily run a backup script where s3cmd checks a folder with over one hundred thousand files to see if any have changed (which by the way, ...
0
votes
0answers
22 views

Are blocked requests by S3 CORS billable?

I'm getting a large amount of POST requests to my S3 bucket. Ideally I'd like to block them all, so I've been looking into CORS. If a request is being blocked by the CORS policy, do I still pay for ...
0
votes
0answers
23 views

Is there a Policy template standard for secure S3 access?

I've created a policy below which restricts SSL access, and requires SSE. Is there a 'standard' policy template recommended by long time AWS S3 devs for locking down S3 buckets that is commonly ...
0
votes
0answers
28 views

How to make amazon s3 files fast from anywhere in the planet?

I'm working on a project with user created content, users basically upload long mp3 files to the server which can then be played by other users. Recently users not from europe reported slow "seek ...
0
votes
0answers
91 views

Force hadoop to use for S3 connection

I'm trying to upload file to S3 using hadoop: hadoop fs -Dfs.s3a.connection.ssl.enabled=false -Dfs.s3a.proxy.host=127.0.0.1 -Dfs.s3a.proxy.port=8123 -put pig_1421167148680.log s3a://access:secret@...
0
votes
0answers
33 views

How to get nginx host variable to work with proxy_pass

I am trying to setup an nginx proxy in front of any s3 website that I point at it. In the example below I have my DNS records customsite.com pointing to my proxy. When I don't use any variable for the ...
0
votes
0answers
32 views

Jenkins aws elastic deployment plugin just create a new ec2 on each update

Here is something strange happening when I tries to deploy from Jenkins to Elastic Beanstalk , you can check my previous question for more clarity. http://stackoverflow.com/questions/34156963/the-...
0
votes
0answers
124 views

S3FS and Fuse, scale out or scale up

I have an infra where I have multiple machines (eg FTP servers, get/put and Samba/NFS servers) that need to share out the same (HA) storage. So I figured to use s3fs and fuse. I got this all setup and ...
0
votes
0answers
80 views

How to use Amazon S3 Ceph Interface on Windows

I have 4 ceph nodes running on virtual machines in our corporate network. I've installed a ceph object gateway on admin node. Can I somehow use it in Amazon S3 style from my windows machine in the ...
0
votes
0answers
122 views

Amazon S3 upload error: An exception occurred while uploading parts to a multipart upload

I am trying to upload a 30 GB file to Amazon S3 using the AWS PHP SDK. require('../vendor/autoload.php'); $client = new Aws\S3\S3Client([ 'version' => 'latest', 'region' => 'us-east-1' ]); $...
0
votes
0answers
85 views

How Do I Mask the CDN Domain with My Own Domain Name?

So i've been trying out cloudfront from AWS. Why is the URL 'example.com' resolving to 'example.com.s3.eu-west-1.amazonaws.com' in the Address Bar? I've tried googling loads of questions with this ...