This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

4
votes
1answer
378 views

Amazon S3 appears to be blocking cloudflare IP addresses. How do I fix it?

I have static content that is being served by Cloudflare. Cloudflare points to Amazon S3 to pick up the static content and serves it via a CNAME (cdn.mydomain.com). The bucket is setup properly and ...
3
votes
1answer
85 views

Deny anonymous downloads in AWS S3

I have a bucket with several files publicly downloadable. I want to put up a bucket policy after which these files should only be downloadable by my IAM users. The policy I have got so far is this: { ...
3
votes
1answer
3k views

s3put fails with “Connection reset by peer” for large files

I can upload small files quite fine using the same commands, however whenever I try to upload a 5 GB file to S3 then the s3 script fails: [backup]$ s3put --bucket john-murdoch-bucket --reduced ...
2
votes
1answer
331 views

Postgresql WAL-E - Replication falling behind + can't catch up

Does anyone know how I can have my replications process the WAL segments faster while using WAL-E? https://github.com/wal-e/wal-e I have one master that leverages WAL-E to upload to S3, and I have 6 ...
2
votes
1answer
534 views

How backup files with logrotate and Amazon S3?

I would like to create a new standard of two things: how long would be generated and be rotated the logs generated by my applications ? how to transfer the logs to Amazon S3, as a backup server ?. ...
2
votes
1answer
846 views

Best failover strategy for e-mail servers on AWS to ensure high availability

We have our e-mail server hosted on AWS. Last week Amazon had a failure in their East Coast region which brought down our server along with many others. We now want to implement a failover strategy ...
2
votes
1answer
624 views

Amazon S3 Website & MySQL Backup

I have a website (a digital asset management system/gallery - http://www.resourcespace.org) that has a huge amount of images. The total size of the website, including the images is approximately 6gb. ...
2
votes
1answer
531 views

How to access an Eucalyptus Walrus(S3) using Amazon AWS SDK

I have downloaded the Amazon AWS SDK for C#, I have no problem accessing the EC2 part of our private cloud running Eucalyptus, I can list, Images, Instances, Zones ... This is working fine : ...
1
vote
1answer
97 views

Nginx proxy cache invalidating request to s3

I have set up a proxy from my application server to a private s3 bucket to cache requests. I was having some trouble with it where s3 was rejecting my download requests (403 forbidden) and after some ...
1
vote
1answer
181 views

Backing up onto S3 - do I want to use bucket in a different region?

We are using Amanda to send backups to S3. Each host is acting as its own Amanda "server". All hosts are currently using the same S3-bucket -- each has its own subfolder under it. The dumps are saved ...
1
vote
1answer
636 views

Cloud-front backed with Nginx (which proxies to S3) randomly missing already cached items?

I wish to serve images from a S3 bucket with Cloudfront as CDN frontend, for that I tried the following: What I wish to acheive (Attempt 2) -- (Misses cloudfront cache randomly) I have the ...
1
vote
1answer
605 views

Debugging logrotate postrotate script

Following is my logrotate conf. /mnt/je/logs/apache/jesites/web/*.log" { missingok rotate 0 size 5M copytruncate notifempty sharedscripts ...
1
vote
1answer
270 views

Restore snapshot from other aws account?

I am new in this, so forgive my beginner question. I have an aws account which is currently in us-west-2c. It has a instance running, and also a couple of volumes which i made. I have multiple backups ...
1
vote
1answer
935 views

Using TntDrive to mount Amazon S3 on a Windows Amazon EC2 instance

I currently have 2 Amazon EC2 Windows Server 2008 instances setup with a load balancer. My problem now is that I need both instances to be able to access a single file system source (I'm trying to use ...
1
vote
1answer
692 views

Protected bucket @ S3 - Access Denied

I'm trying to grant RW access to a given bucket to a specific user using the following bucket policy: { "Id": "Policy1322043790167", "Statement": [ { "Sid": "Stmt9999043784080", ...
2
votes
0answers
56 views

Solution for file data backup, versioning and replication

We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
2
votes
0answers
136 views

s3 policy for quota

Is there a s3 policy rule that can only allow a user to upload so many GB of files? Basically looking to set a quota on the objects a user can upload. Here is the rule set that I have now: { ...
2
votes
0answers
398 views

How can I whitelist oubound-from-private-subnet traffic to S3 on the NAT instance of an EC2 VPC?

I'm researching privacy-compliant high-security deployment options for AWS VPC. I'm looking for a method of tightening outbound traffic from the private subnets at the NAT instance. For example, how ...
2
votes
0answers
213 views

Gluster with s3ql mounted brick?

I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount. Having just the plain directory as a unique ...
2
votes
0answers
337 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
1
vote
0answers
32 views

AWS S3 bucket permissions

I've created AWS s3 buckets with several thousand objects, initially the access policy was restricted to authenticated users, but later due to some requirement we need to make all objects in this ...
1
vote
0answers
37 views

jboss cluster in ec2 with s3ping

Am trying to get s3ping discovery method working in wildfly 8.0.0. I am using default Amazon Linux AMI and using chef to launch instances in ec2 based on this tutorial ...
1
vote
0answers
248 views

s3cmd sync hangs forever. Never finishes

I am trying to backup a directory of around 350GB in size to S3. The command I run is s3cmd --progress -v -H sync /localdir s3://mybucket/ However, the process never moves past INFO: Compiling ...
1
vote
0answers
50 views

svnadmin dump returns error insufficient disk space

I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked. I have Apache Subversion hosted on an ...
1
vote
0answers
180 views

Duplicity -> S3 backup failing with “[Errno 105] No buffer space available”

I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
1
vote
0answers
362 views

Amazon S3: allow user access to subfolder only on any bucket

I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket. This is my policy so far (reading works, but it ...
1
vote
0answers
103 views

Handling multiple server blocks in nginx configuration

I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com. Eg:If ...
1
vote
0answers
185 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
1
vote
0answers
990 views

How can I move s3 objects >5gb from one bucket to another?

Objects larger than 5GB are supported with multi-part uploading. We have 5gb+ files in one bucket already that I'd like to move to another, under the same AWS account. When I issue the command using ...
1
vote
0answers
296 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
0
votes
0answers
13 views

Nginx as s3 proxy with private buckets

I'm trying to set up Nginx as a proxy for a static html/js webapp hosted on S3. I read a lots of tutorials and posts and I could make it work if my bucket is set as public. The reason I need Nginx as ...
0
votes
0answers
12 views

How to stop s3 from listing some of the content?

It is an example. What setting of S3 causes the s3 server to a listing of bucket content?
0
votes
0answers
19 views

duplicity s3 backup from centos

I'm trying to backup server data to S3 but getting no success. Below is the script which I run to start backup on S3. I have create a user using IAM and have granted all permissions to the user. ...
0
votes
0answers
5 views

How can I improve the space usage of duplicate content in S3?

In my project I have duplicated files in various folders in a S3 bucket. Based on the output of s3cmd du, each time I make a copy of some content in S3, the usage increases. I wonder does s3 support ...
0
votes
0answers
5 views

Read data from external bucket on Amazon EMR

I need to read data from a bucket that I don't own, and I get the IAM account key and secrete access key to the bucket. What is the best way I can access the data? How can I add another IAM with ...
0
votes
0answers
71 views

AWS Cloud Front with S3, serving static files- Redirects implemented via java script at browser level

We have our Java application on tomcat server webpages- the application fetch content from multiple services and Database to create page and respond. We are checking for a best method to improve the ...
0
votes
0answers
95 views

Override content type headers when using Apache as a reverse proxy

I'm using Apache 2.2 as a reverse proxy in front of my s3 buckets. S3 serve the hosted files with wrong content-type headers. Is there a way to configure Apache to automatically re write the ...
0
votes
0answers
232 views

IAM user with full s3 access can't create directory

I've a AWS IAM user which has full access to Amazon S3 The policy is the following { "Statement": [ { "Effect": "Allow", "Action": "s3:ListAllMyBuckets", ...
0
votes
0answers
65 views

NGINX: proxy_cache altering headers

I have been having trouble building a proxy between my server and s3 to cache requests. I finally boiled it down to a single point of failure... http{ proxy_cache_path /home/cache levels=1:2 ...
0
votes
0answers
43 views

Benchmark network (internet) connection on server

I have a dedicated ubuntu server I rentet. Now I'm running a lot of ruby background processing on that server, that - among other things - does a lot of get requests to very small images (about 10kb) ...
0
votes
0answers
20 views

Include x-amz-storage-class Header When Calling Amazon S3 from Web.config

Any time my web app calls Amazon S3, it would be nice to conditionally include the following header. I am using a custom domain mapped to an S3 bucket. key: x-amz-storage-class value: ...
0
votes
0answers
448 views

s3fs “could not determine how to establish security credentials”

I'm trying to use s3fs to mount an S3 bucket on to a standard AWS Amazon Linux AMI (with all the necessary dependencies installed). However when following this tutorial when I run: s3fs mybucketname ...
0
votes
0answers
79 views

Strict-Transport-Security on CloudFront with S3 origin?

My company's site has a static homepage for speed and cost reasons. We use S3 as the origin for CloudFront. Now, we would like to declare Strict-Transport-Security for the entire domain, but S3 seems ...
0
votes
0answers
48 views

Move query string to path in proxy - Apache

I am setting up Apache to proxy files from Amazon S3. The initial urls have query strings, but query strings can't be passed to S3, so they are stored as paths with question marks in them (which then ...
0
votes
0answers
59 views

Nginx serves response with 200 even after calling ngx.exit()

I am trying to serve static content from S3. I am using nginx and lua script for its configuration. I am new to both nginx and lua. The task that I want to achieve is : Get request URL into nginx. ...
0
votes
0answers
206 views

Cloudfront - How to invalidate objects in a distribution that was transformed from secured to public?

The setting I have an Amazon Cloudfront distribution that was originally set as secured. Objects in this distribution required a URL signing. For example, a valid URL used to be of the following ...
0
votes
0answers
293 views

Duplicati restore one file from s3

I am attempting to configure Duplicati to backup encrypted to Amazon S3. I configured it to run the CLI in a batch file that will be TaskSched'd to run nightly. This batch code seems to be working: ...
0
votes
0answers
132 views

Linux server backup to Amazon S3 using SSL

I have Linux box CentOS 6.5 and i want to backup daily files like SQL dumps and config files to Amazon S3 I have installed Ruby and S3Sync. But the problem when running the s3sync with these ...
0
votes
0answers
952 views

s3cmd 403 Access Denied error on put but not ls

I can access s3 using s3cmd on an ec2 instance to do s3cmd ls s3://myc bucketname but not using 'put' to add a file. It gives a 403 access denied error. I am using IAM and have group setup to access ...
0
votes
0answers
479 views

s3cmd fails to put files

I'm trying to put mysql dump file on s3 bucket. It gives an error like ; sudo ./backup.sh WARNING: Module python-magic is not available. Guessing MIME types based on file extensions. ...