This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

3
votes
0answers
141 views

Set up a different index document per directory with S3 static website hosting

I am using S3 static website hosting, and trying to get the following redirects to work: example.com --> example.com/index.html example.com/foo --> example.com/foo/foo.html example.com/bar ...
3
votes
0answers
58 views

S3 + Cloudflare vs S3 + 2 Web Server + Cloudflate

Currently, I run two websites and each site own it own big server. Currently, all of those two sites has roughly 300GB of images. Its grow rate is 15GB of images per month. I know that eventually, one ...
3
votes
0answers
384 views

Strict-Transport-Security on CloudFront with S3 origin?

My company's site has a static homepage for speed and cost reasons. We use S3 as the origin for CloudFront. Now, we would like to declare Strict-Transport-Security for the entire domain, but S3 seems ...
2
votes
0answers
537 views

AWS CLI throws “Unable to locate credentials”, the second time it's run

I'm trying to download some files from S3 to an EC2 instance using the AWS CLI (1.8.7) on startup using a User-Data Script. This instance has a IAM Role & Instance Profile with the appropriate ...
2
votes
0answers
774 views

Nginx as s3 proxy with private buckets

I'm trying to set up Nginx as a proxy for a static html/js webapp hosted on S3. I read a lots of tutorials and posts and I could make it work if my bucket is set as public. The reason I need Nginx as ...
2
votes
0answers
110 views

Solution for file data backup, versioning and replication

We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
2
votes
0answers
2k views

How can I move s3 objects >5gb from one bucket to another?

Objects larger than 5GB are supported with multi-part uploading. We have 5gb+ files in one bucket already that I'd like to move to another, under the same AWS account. When I issue the command using ...
2
votes
0answers
237 views

Gluster with s3ql mounted brick?

I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount. Having just the plain directory as a unique ...
2
votes
0answers
514 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
1
vote
0answers
22 views

fast backup script in linux

I have a server with more applications and do normal shell command to take backups. it took couple of days to complete. Tried to install duplicity. but its told some packages are not found. its Debian ...
1
vote
0answers
23 views

Is there a way to get the size of Amzon S3 bucket from S3 console?

I wonder if there is some way to get the size of Amazon S3 bucket directly from console https://console.aws.amazon.com/s3/ Clicking properties returns great deal of information, but not the size of ...
1
vote
0answers
24 views

Bucket policy that respects pre-signed URLs OR IP Address deny?

I would like to be able to restrict access to files in a S3 bucket in multiple ways. This is due to the fact that the files stored can be accessed in different manners. We do this because we have TBs ...
1
vote
0answers
126 views

aws s3 sync: how to exclude proc, sys directories

I can't seem to get the aws sync s3 command to exclude directories. I am backing up a mounted filesystem from OpenVZ which is located at /vz/root/100. My command is: aws s3 sync /vz/root/100 ...
1
vote
0answers
662 views

s3cmd sync hangs forever. Never finishes

I am trying to backup a directory of around 350GB in size to S3. The command I run is s3cmd --progress -v -H sync /localdir s3://mybucket/ However, the process never moves past INFO: Compiling ...
1
vote
0answers
56 views

svnadmin dump returns error insufficient disk space

I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked. I have Apache Subversion hosted on an ...
1
vote
0answers
228 views

Duplicity -> S3 backup failing with “[Errno 105] No buffer space available”

I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
1
vote
0answers
597 views

Amazon S3: allow user access to subfolder only on any bucket

I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket. This is my policy so far (reading works, but it ...
1
vote
0answers
132 views

Handling multiple server blocks in nginx configuration

I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com. Eg:If ...
1
vote
0answers
261 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
1
vote
0answers
343 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
0
votes
0answers
28 views

Reverse Proxy on cloudfront

I have an angular site / web application (Lets call this www.mysite.com) hosted on Amazon S3 as a static website. This s3 bucket serves as the origin for a cloudfront distribution and it works ...
0
votes
0answers
16 views

Enabling S3 bucket as website using cli

I figured out how to create an S3 bucket using aws command line. Is it possible to enable the s3 bucket I created as website using aws cli? The reason I'm doing this is because we're automating ...
0
votes
0answers
13 views

AWS S3 “service user” for non-AWS server?

I'm setting up a server outside of AWS, but I'd like it to read and write from one of my AWS S3 buckets. I know that within AWS, best practice is to assign a role with S3 read/write permission to the ...
0
votes
0answers
26 views

AWS CLI copy life-cycle configuration between buckets

I am trying to read-modify-write an existing life-cycle configuration. So the simpler thing I am staring out with is copying a life-cycle configuration between buckets. From reading through the ...
0
votes
0answers
13 views

Google Storage Sync from Amazon S3 is not running

I setup a new Google Storage bucket, then added the username and password from the S3 permissions, including the correct policy as instructed on Amazon S3 and then scheduled the sync for 3 hours in ...
0
votes
0answers
18 views

Syncing from a NAS drive to AWS S3 with s3cmd

I'm trying to use s3cmd to sync data from a NAS drive connected to a web server node, to AWS S3. I'm using this command: s3cmd sync ./logs/ s3://bucket/logs/ What's happening is there is a massive ...
0
votes
0answers
52 views

Nginx Proxy pass with aws s3 gives 403

I have an web app that uploads images in server and copies them to s3 bucket using awscli. Nginx on server retrieves images from s3 through reverse proxy. When I try to open image using url, it gives ...
0
votes
0answers
31 views

Read Files from S3FS mount with PHP

We have a S3 bucket we use for video uploading. I also have a script for accessing the Youtube API so I can pull the upload files from a server rather than via the Web interface. This used to work ...
0
votes
0answers
25 views

When s3cmd checks a bucket with thousands of files to sync it, does this count as one list request or thousands?

Trying to figure out pricing here so I don't get overcharged. I daily run a backup script where s3cmd checks a folder with over one hundred thousand files to see if any have changed (which by the way, ...
0
votes
0answers
19 views

Are blocked requests by S3 CORS billable?

I'm getting a large amount of POST requests to my S3 bucket. Ideally I'd like to block them all, so I've been looking into CORS. If a request is being blocked by the CORS policy, do I still pay for ...
0
votes
0answers
22 views

Is there a Policy template standard for secure S3 access?

I've created a policy below which restricts SSL access, and requires SSE. Is there a 'standard' policy template recommended by long time AWS S3 devs for locking down S3 buckets that is commonly ...
0
votes
0answers
22 views

How to make amazon s3 files fast from anywhere in the planet?

I'm working on a project with user created content, users basically upload long mp3 files to the server which can then be played by other users. Recently users not from europe reported slow "seek ...
0
votes
0answers
55 views

Force hadoop to use for S3 connection

I'm trying to upload file to S3 using hadoop: hadoop fs -Dfs.s3a.connection.ssl.enabled=false -Dfs.s3a.proxy.host=127.0.0.1 -Dfs.s3a.proxy.port=8123 -put pig_1421167148680.log ...
0
votes
0answers
28 views

How to get nginx host variable to work with proxy_pass

I am trying to setup an nginx proxy in front of any s3 website that I point at it. In the example below I have my DNS records customsite.com pointing to my proxy. When I don't use any variable for the ...
0
votes
0answers
27 views

Jenkins aws elastic deployment plugin just create a new ec2 on each update

Here is something strange happening when I tries to deploy from Jenkins to Elastic Beanstalk , you can check my previous question for more clarity. ...
0
votes
0answers
88 views

S3FS and Fuse, scale out or scale up

I have an infra where I have multiple machines (eg FTP servers, get/put and Samba/NFS servers) that need to share out the same (HA) storage. So I figured to use s3fs and fuse. I got this all setup and ...
0
votes
0answers
80 views

ERROR — : Response was: 307 — Temporary Redirect when trying to upload to bucket

I get 307 when uploading to a bucket with Amazon's Ruby gem (2.10.2). Another bucket works fine (both were created today). I checked the name and I didn't misspell it. What is causing the error?
0
votes
0answers
65 views

How to use Amazon S3 Ceph Interface on Windows

I have 4 ceph nodes running on virtual machines in our corporate network. I've installed a ceph object gateway on admin node. Can I somehow use it in Amazon S3 style from my windows machine in the ...
0
votes
0answers
107 views

Amazon S3 upload error: An exception occurred while uploading parts to a multipart upload

I am trying to upload a 30 GB file to Amazon S3 using the AWS PHP SDK. require('../vendor/autoload.php'); $client = new Aws\S3\S3Client([ 'version' => 'latest', 'region' => 'us-east-1' ]); ...
0
votes
0answers
59 views

How Do I Mask the CDN Domain with My Own Domain Name?

So i've been trying out cloudfront from AWS. Why is the URL 'example.com' resolving to 'example.com.s3.eu-west-1.amazonaws.com' in the Address Bar? I've tried googling loads of questions with this ...
0
votes
0answers
71 views

looking for index.php while proxy_pass to s3 bucket

I am configuring my nginx proxy server to fetch data from s3 bucket in case of 502 error. This is my configuration file. error_page 502 @static; location @static{ rewrite ^/a/b /a_b break; ...
0
votes
0answers
159 views

CloudFront + S3 Access Denied Using Signed Cookies

I'm trying to integrate signed cookies for a CloudFront distribution that's fed off of an S3 bucket but getting access denied error messages: <Error> <Code>AccessDenied</Code> ...
0
votes
0answers
158 views

Access denied for the request of data from s3 bucket on high load

I am trying to fetch static files from s3 bucket in case of 502 error. This is how I have configured nginx for requests : error_page 502 = @static; location / { index index.php; try_files $uri ...
0
votes
0answers
70 views

Amazon S3 - Upfront File Organization Architectural Decisions

I'm currently building a web application for a publishing platform, and I am using S3 for content storage. S3 is super simple to use, but I'd like to try to make the best possible file organization ...
0
votes
0answers
33 views

How to use the s3cmd setlifecycle command?

s3cmd has a setlifecycle subcommand that seems like it ought to let me configure a bucket into Amazon Glacier. However I can't find any documentation for it on the web or in the manpage. What's the ...
0
votes
0answers
40 views

aws hosted website not serving web pages to internal network and serving from s3 bucket instead

Sorry about more text to describe..(The issue has been resolved but the causes are un-known) We Have one website hosted via AWS Load Balancer and was working from last year perfectly. Recently an ...
0
votes
0answers
38 views

How to securely allow web apps to read/write to S3? (the new way with IAM)

I've been banging my head against a wall on this for two hours. It's frustrating that Amazon has made this so complicated when it doesn't need to be. Let's say I have a bucket called "test," and a ...
0
votes
0answers
117 views

DNS Based Load balancing on squid - odd issues

I'm attempting to do round-robin DNS based load balancing for two squid boxes running as simple HTTP(S) proxies. For testing I'm using AWS Route 53. To simulate round-robin I created two records ...
0
votes
0answers
672 views

AWS security group egress rules for S3

I manually created a new security group using the AWS CLI. I created ingress rules that allow incoming connections only from my company's public IP address using the known ports for SSH (22) and ...
0
votes
0answers
75 views

Duplicity GPG key not available

I am using the following tutorial to get Duplicity to back up my mysql dumps to Amazon S3. http://kappataumu.com/articles/cloud-backups-duplicity-s3.html However I am getting an issue with the ...