This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

4
votes
1answer
349 views

Amazon S3 appears to be blocking cloudflare IP addresses. How do I fix it?

I have static content that is being served by Cloudflare. Cloudflare points to Amazon S3 to pick up the static content and serves it via a CNAME (cdn.mydomain.com). The bucket is setup properly and ...
2
votes
1answer
288 views

Postgresql WAL-E - Replication falling behind + can't catch up

Does anyone know how I can have my replications process the WAL segments faster while using WAL-E? https://github.com/wal-e/wal-e I have one master that leverages WAL-E to upload to S3, and I have 6 ...
2
votes
1answer
432 views

How backup files with logrotate and Amazon S3?

I would like to create a new standard of two things: how long would be generated and be rotated the logs generated by my applications ? how to transfer the logs to Amazon S3, as a backup server ?. ...
2
votes
1answer
598 views

Amazon S3 Website & MySQL Backup

I have a website (a digital asset management system/gallery - http://www.resourcespace.org) that has a huge amount of images. The total size of the website, including the images is approximately 6gb. ...
2
votes
1answer
517 views

How to access an Eucalyptus Walrus(S3) using Amazon AWS SDK

I have downloaded the Amazon AWS SDK for C#, I have no problem accessing the EC2 part of our private cloud running Eucalyptus, I can list, Images, Instances, Zones ... This is working fine : ...
1
vote
1answer
110 views

Backing up onto S3 - do I want to use bucket in a different region?

We are using Amanda to send backups to S3. Each host is acting as its own Amanda "server". All hosts are currently using the same S3-bucket -- each has its own subfolder under it. The dumps are saved ...
1
vote
1answer
602 views

AWS Cloudfront multiple origins - how can we fallback to 2nd origin if not found in the 1st

We're trying to set up a Cloudfront distribution where the default origin is S3, but if the content is not found on S3, we want it to fetch it from an EC2 instance. Can we setup these multiple ...
1
vote
1answer
550 views

Debugging logrotate postrotate script

Following is my logrotate conf. /mnt/je/logs/apache/jesites/web/*.log" { missingok rotate 0 size 5M copytruncate notifempty sharedscripts ...
1
vote
1answer
210 views

Restore snapshot from other aws account?

I am new in this, so forgive my beginner question. I have an aws account which is currently in us-west-2c. It has a instance running, and also a couple of volumes which i made. I have multiple backups ...
1
vote
1answer
765 views

Best failover strategy for e-mail servers on AWS to ensure high availability

We have our e-mail server hosted on AWS. Last week Amazon had a failure in their East Coast region which brought down our server along with many others. We now want to implement a failover strategy ...
1
vote
1answer
849 views

Using TntDrive to mount Amazon S3 on a Windows Amazon EC2 instance

I currently have 2 Amazon EC2 Windows Server 2008 instances setup with a load balancer. My problem now is that I need both instances to be able to access a single file system source (I'm trying to use ...
1
vote
1answer
648 views

Protected bucket @ S3 - Access Denied

I'm trying to grant RW access to a given bucket to a specific user using the following bucket policy: { "Id": "Policy1322043790167", "Statement": [ { "Sid": "Stmt9999043784080", ...
1
vote
1answer
352 views

Reducing latency for different geographic regions on Amazon Cloud

I have got an application which has three components Application code : Amazon EC2 US-EAST-1 instance Application images, and other static data : Amazon S3 with CloudFront Application Database : ...
0
votes
1answer
145 views

Setting cache for already uploaded amazon s3 files

In amazon s3 for our site through s3cmd tool we uploaded around 60GB of files including mp3, jpg to a bucket. But now when we checked site slowness files are not caching and further checking we came ...
0
votes
1answer
90 views

Duplicity Backups (via Duply) to S3 - high server load

We have been doing incremental backups using Duply on our main server, to an S3 bucket. However, we've found that there is a high server load during the backups (its an Amazon EC2 server). We're ...
0
votes
1answer
132 views

rebuild yum index on aws s3

I am trying to rebuild yum repo on aws S3 after adding new packages. Here are few commands I am trying, but it is not helping. [root@chucks ~]$ createrepo --baseurl ...
3
votes
0answers
2k views

s3put fails with “Connection reset by peer” for large files

I can upload small files quite fine using the same commands, however whenever I try to upload a 5 GB file to S3 then the s3 script fails: [backup]$ s3put --bucket john-murdoch-bucket --reduced ...
2
votes
0answers
48 views

Deny anonymous downloads in AWS S3

I have a bucket with several files publicly downloadable. I want to put up a bucket policy after which these files should only be downloadable by my IAM users. The policy I have got so far is this: { ...
2
votes
0answers
29 views

Solution for file data backup, versioning and replication

We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
2
votes
0answers
70 views

s3 policy for quota

Is there a s3 policy rule that can only allow a user to upload so many GB of files? Basically looking to set a quota on the objects a user can upload. Here is the rule set that I have now: { ...
2
votes
0answers
347 views

How can I whitelist oubound-from-private-subnet traffic to S3 on the NAT instance of an EC2 VPC?

I'm researching privacy-compliant high-security deployment options for AWS VPC. I'm looking for a method of tightening outbound traffic from the private subnets at the NAT instance. For example, how ...
2
votes
0answers
207 views

Gluster with s3ql mounted brick?

I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount. Having just the plain directory as a unique ...
2
votes
0answers
312 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
1
vote
0answers
151 views

s3cmd sync hangs forever. Never finishes

I am trying to backup a directory of around 350GB in size to S3. The command I run is s3cmd --progress -v -H sync /localdir s3://mybucket/ However, the process never moves past INFO: Compiling ...
1
vote
0answers
46 views

svnadmin dump returns error insufficient disk space

I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked. I have Apache Subversion hosted on an ...
1
vote
0answers
171 views

Duplicity -> S3 backup failing with “[Errno 105] No buffer space available”

I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
1
vote
0answers
320 views

Amazon S3: allow user access to subfolder only on any bucket

I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket. This is my policy so far (reading works, but it ...
1
vote
0answers
94 views

Handling multiple server blocks in nginx configuration

I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com. Eg:If ...
1
vote
0answers
165 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
1
vote
0answers
276 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
0
votes
0answers
45 views

AWS Cloud Front with S3, serving static files- Redirects implemented via java script at browser level

We have our Java application on tomcat server webpages- the application fetch content from multiple services and Database to create page and respond. We are checking for a best method to improve the ...
0
votes
0answers
69 views

Override content type headers when using Apache as a reverse proxy

I'm using Apache 2.2 as a reverse proxy in front of my s3 buckets. S3 serve the hosted files with wrong content-type headers. Is there a way to configure Apache to automatically re write the ...
0
votes
0answers
35 views

AWSCLI s3 sync doesn't create directory

Hi try to sync a local directory to a S3 bucket. The files in the directory get synced perfectly, just the directory isn't created at all. Can you guys give me a hint how to do it? "newdir" contains ...
0
votes
0answers
89 views

IAM user with full s3 access can't create directory

I've a AWS IAM user which has full access to Amazon S3 The policy is the following { "Statement": [ { "Effect": "Allow", "Action": "s3:ListAllMyBuckets", ...
0
votes
0answers
52 views

NGINX: proxy_cache altering headers

I have been having trouble building a proxy between my server and s3 to cache requests. I finally boiled it down to a single point of failure... http{ proxy_cache_path /home/cache levels=1:2 ...
0
votes
0answers
75 views

Nginx proxy cache invalidating request to s3

I have set up a proxy from my application server to a private s3 bucket to cache requests. I was having some trouble with it where s3 was rejecting my download requests (403 forbidden) and after some ...
0
votes
0answers
35 views

Benchmark network (internet) connection on server

I have a dedicated ubuntu server I rentet. Now I'm running a lot of ruby background processing on that server, that - among other things - does a lot of get requests to very small images (about 10kb) ...
0
votes
0answers
18 views

Include x-amz-storage-class Header When Calling Amazon S3 from Web.config

Any time my web app calls Amazon S3, it would be nice to conditionally include the following header. I am using a custom domain mapped to an S3 bucket. key: x-amz-storage-class value: ...
0
votes
0answers
314 views

s3fs “could not determine how to establish security credentials”

I'm trying to use s3fs to mount an S3 bucket on to a standard AWS Amazon Linux AMI (with all the necessary dependencies installed). However when following this tutorial when I run: s3fs mybucketname ...
0
votes
0answers
49 views

Strict-Transport-Security on CloudFront with S3 origin?

My company's site has a static homepage for speed and cost reasons. We use S3 as the origin for CloudFront. Now, we would like to declare Strict-Transport-Security for the entire domain, but S3 seems ...
0
votes
0answers
37 views

Move query string to path in proxy - Apache

I am setting up Apache to proxy files from Amazon S3. The initial urls have query strings, but query strings can't be passed to S3, so they are stored as paths with question marks in them (which then ...
0
votes
0answers
48 views

Nginx serves response with 200 even after calling ngx.exit()

I am trying to serve static content from S3. I am using nginx and lua script for its configuration. I am new to both nginx and lua. The task that I want to achieve is : Get request URL into nginx. ...
0
votes
0answers
143 views

Cloudfront - How to invalidate objects in a distribution that was transformed from secured to public?

The setting I have an Amazon Cloudfront distribution that was originally set as secured. Objects in this distribution required a URL signing. For example, a valid URL used to be of the following ...
0
votes
0answers
231 views

Duplicati restore one file from s3

I am attempting to configure Duplicati to backup encrypted to Amazon S3. I configured it to run the CLI in a batch file that will be TaskSched'd to run nightly. This batch code seems to be working: ...
0
votes
0answers
117 views

Linux server backup to Amazon S3 using SSL

I have Linux box CentOS 6.5 and i want to backup daily files like SQL dumps and config files to Amazon S3 I have installed Ruby and S3Sync. But the problem when running the s3sync with these ...
0
votes
0answers
774 views

s3cmd 403 Access Denied error on put but not ls

I can access s3 using s3cmd on an ec2 instance to do s3cmd ls s3://myc bucketname but not using 'put' to add a file. It gives a 403 access denied error. I am using IAM and have group setup to access ...
0
votes
0answers
328 views

AWS Cloudformation Bootstraping with S3 File and Autoscaling

I just started trying to use CloudFormation and am using it for some autoscaling. I am new to all this so bare with me. I am using the sample template for autoscaling found here. ...
0
votes
0answers
443 views

s3cmd fails to put files

I'm trying to put mysql dump file on s3 bucket. It gives an error like ; sudo ./backup.sh WARNING: Module python-magic is not available. Guessing MIME types based on file extensions. ...
0
votes
0answers
805 views

How can I move s3 objects >5gb from one bucket to another?

Objects larger than 5GB are supported with multi-part uploading. We have 5gb+ files in one bucket already that I'd like to move to another, under the same AWS account. When I issue the command using ...
0
votes
0answers
258 views

Getting a simple list of changed files from unison

I have a filesystem that is changed on two servers and also needs to be replicated to Amazon S3. Until recently, syncing the filesystem between the two servers using Unison, and then copying to S3 ...