This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

4
votes
1answer
216 views

Amazon S3 appears to be blocking cloudflare IP addresses. How do I fix it?

I have static content that is being served by Cloudflare. Cloudflare points to Amazon S3 to pick up the static content and serves it via a CNAME (cdn.mydomain.com). The bucket is setup properly and ...
3
votes
1answer
129 views

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which ...
2
votes
1answer
164 views

Postgresql WAL-E - Replication falling behind + can't catch up

Does anyone know how I can have my replications process the WAL segments faster while using WAL-E? https://github.com/wal-e/wal-e I have one master that leverages WAL-E to upload to S3, and I have 6 ...
2
votes
1answer
296 views

How backup files with logrotate and Amazon S3?

I would like to create a new standard of two things: how long would be generated and be rotated the logs generated by my applications ? how to transfer the logs to Amazon S3, as a backup server ?. ...
2
votes
1answer
488 views

Amazon S3 Website & MySQL Backup

I have a website (a digital asset management system/gallery - http://www.resourcespace.org) that has a huge amount of images. The total size of the website, including the images is approximately 6gb. ...
2
votes
1answer
476 views

How to access an Eucalyptus Walrus(S3) using Amazon AWS SDK

I have downloaded the Amazon AWS SDK for C#, I have no problem accessing the EC2 part of our private cloud running Eucalyptus, I can list, Images, Instances, Zones ... This is working fine : ...
1
vote
1answer
31 views

Backing up onto S3 - do I want to use bucket in a different region?

We are using Amanda to send backups to S3. Each host is acting as its own Amanda "server". All hosts are currently using the same S3-bucket -- each has its own subfolder under it. The dumps are saved ...
1
vote
1answer
781 views

How to copy all s3 buckets to another AWS account?

I'm copying all EC2 instances to another AWS account (creating the AMIs, sharing them with that account and running the instances from AMIs there). I also need to move my S3 buckets since the ...
1
vote
1answer
313 views

AWS Cloudfront multiple origins - how can we fallback to 2nd origin if not found in the 1st

We're trying to set up a Cloudfront distribution where the default origin is S3, but if the content is not found on S3, we want it to fetch it from an EC2 instance. Can we setup these multiple ...
1
vote
1answer
150 views

Amazon EC2, S3, REST API, and how to correctly provide contents to users

I have deployed a REST server in an Amazon EC2 instance. I have also configured an Amazon S3 bucket to store all the data generated by users while interacting with the API. The main information stored ...
1
vote
1answer
261 views

Debugging logrotate postrotate script

Following is my logrotate conf. /mnt/je/logs/apache/jesites/web/*.log" { missingok rotate 0 size 5M copytruncate notifempty sharedscripts ...
1
vote
1answer
131 views

Restore snapshot from other aws account?

I am new in this, so forgive my beginner question. I have an aws account which is currently in us-west-2c. It has a instance running, and also a couple of volumes which i made. I have multiple backups ...
1
vote
1answer
225 views

Taking Cassandra Backups

We currently have 12 nodes running in our Cassandra cluster. Ultimately even if a couple of the nodes go down, we're still up and running. The paranoia in me would like to do at least one backup a day ...
1
vote
1answer
562 views

Best failover strategy for e-mail servers on AWS to ensure high availability

We have our e-mail server hosted on AWS. Last week Amazon had a failure in their East Coast region which brought down our server along with many others. We now want to implement a failover strategy ...
1
vote
1answer
662 views

Using TntDrive to mount Amazon S3 on a Windows Amazon EC2 instance

I currently have 2 Amazon EC2 Windows Server 2008 instances setup with a load balancer. My problem now is that I need both instances to be able to access a single file system source (I'm trying to use ...
1
vote
1answer
482 views

Protected bucket @ S3 - Access Denied

I'm trying to grant RW access to a given bucket to a specific user using the following bucket policy: { "Id": "Policy1322043790167", "Statement": [ { "Sid": "Stmt9999043784080", ...
1
vote
1answer
326 views

Reducing latency for different geographic regions on Amazon Cloud

I have got an application which has three components Application code : Amazon EC2 US-EAST-1 instance Application images, and other static data : Amazon S3 with CloudFront Application Database : ...
0
votes
1answer
49 views

Amazon Cloudfront with S3 Redirect

I'm trying to do something that should be relatively straight-forward - I want to setup a few domains and subdomains to redirect to the core domain for our site, but I want to put the redirects out on ...
3
votes
0answers
1k views

Amazon Cloudfront with S3. Access Denied

We're trying to distribute out S3 buckets via Cloudfront but for some reason the only response is an AccessDenied XML document like the following: <Error> ...
3
votes
0answers
876 views

Amazon S3 + Cloudfront VS Rackspace CloudFiles

I know this question came up lot of time on the internet , But I couldn't find any latest updates on this . So I gathered the information what I found and putting here and have few questions on it . ...
2
votes
0answers
179 views

Gluster with s3ql mounted brick?

I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount. Having just the plain directory as a unique ...
2
votes
0answers
269 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
1
vote
0answers
33 views

svnadmin dump returns error insufficient disk space

I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked. I have Apache Subversion hosted on an ...
1
vote
0answers
125 views

Duplicity -> S3 backup failing with “[Errno 105] No buffer space available”

I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
1
vote
0answers
96 views

Drop/Truncate DynamoDB table with AWS Data Pipeline

I have set up a Data Pipeline that imports files from an S3 bucket to a DynamoDB table, based on the predefined example. I want to truncate the table (or drop and create a new one) every time the ...
1
vote
0answers
175 views

Amazon S3: allow user access to subfolder only on any bucket

I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket. This is my policy so far (reading works, but it ...
1
vote
0answers
227 views

How can I whitelist oubound-from-private-subnet traffic to S3 on the NAT instance of an EC2 VPC?

I'm researching privacy-compliant high-security deployment options for AWS VPC. I'm looking for a method of tightening outbound traffic from the private subnets at the NAT instance. For example, how ...
1
vote
0answers
75 views

Handling multiple server blocks in nginx configuration

I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com. Eg:If ...
1
vote
0answers
106 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
1
vote
0answers
2k views

s3put fails with “Connection reset by peer” for large files

I can upload small files quite fine using the same commands, however whenever I try to upload a 5 GB file to S3 then the s3 script fails: [backup]$ s3put --bucket john-murdoch-bucket --reduced ...
1
vote
0answers
247 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
1
vote
0answers
512 views

How to backup MySql to Amazon S3?

What is the best software to use to regularly backup a MySQL database into Amazon S3? It looks like there are several solutions out there, and I am just wondering if there's a consensus on what's ...
0
votes
0answers
6 views

Cloudfront - How to invalidate objects in a distribution that was transformed from secured to public?

The setting I have an Amazon Cloudfront distribution that was originally set as secured. Objects in this distribution required a URL signing. For example, a valid URL used to be of the following ...
0
votes
0answers
9 views

Setting Access-Control-Allow-Origin on Cloudfront

I am having problems serving static assets to Firefox using AWS Cloudfront. Chrome works perfect, but Firefox is returning a CORS error. If I execute curl , I get: HTTP/1.1 200 OK Content-Type: ...
0
votes
0answers
42 views

Duplicati restore one file from s3

I am attempting to configure Duplicati to backup encrypted to Amazon S3. I configured it to run the CLI in a batch file that will be TaskSched'd to run nightly. This batch code seems to be working: ...
0
votes
0answers
62 views

Linux server backup to Amazon S3 using SSL

I have Linux box CentOS 6.5 and i want to backup daily files like SQL dumps and config files to Amazon S3 I have installed Ruby and S3Sync. But the problem when running the s3sync with these ...
0
votes
0answers
35 views

Best practice for web applications and windows services remote configuration in AWS

we have a distributed system that contains windows services and web applications (.NET and Java based). Currently, most of the initial configuration for these applications are taken from various ...
0
votes
0answers
243 views

s3cmd 403 Access Denied error on put but not ls

I can access s3 using s3cmd on an ec2 instance to do s3cmd ls s3://myc bucketname but not using 'put' to add a file. It gives a 403 access denied error. I am using IAM and have group setup to access ...
0
votes
0answers
128 views

AWS Cloudformation Bootstraping with S3 File and Autoscaling

I just started trying to use CloudFormation and am using it for some autoscaling. I am new to all this so bare with me. I am using the sample template for autoscaling found here. ...
0
votes
0answers
183 views

Amazon Powershell Write-S3Object can't resolve s3.amazonaws.com

Am seeing incredibly weird behavior for the Write-S3Object cmdlet provided by Amazon. I've got a backup script running Write-S3Object on my Windows Server 2008 R2 box. When I kick it off, it ...
0
votes
0answers
205 views

Amazon S3 upload files of 2GB using the high level multipart loading sometimes fails

I tried to load files which are close to 2 GB, in code I used Amazon's api for multipart loading. Here is a sample from the code: ObjectMetadata o = new ObjectMetadata(); ...
0
votes
0answers
76 views

nginx passing assets from S3

I'm trying to configure Nginx to passing assets from S3.I have the below in the config but when am trying to access frontend, nginx return 404 on static resources. And nginx error log is empty. Can ...
0
votes
0answers
109 views

haproxy use backend as a parameter

I have S3 bucket which include all my files which refer as version for example file-v1.html file-v2.html and etc ... I want to redirect a requests with haproxy to those file by calling to ..../latest ...
0
votes
0answers
52 views

Restoring WordPress EC2 instance from snapshot results in 403 Forbidden error

This problem has been perplexing me for weeks now. Here's how the issue goes: Launch Amazon Linux 64-bit instance, successfully install WordPress, and site is active w/ no issues Create snapshot ...
0
votes
0answers
246 views

s3cmd fails to put files

I'm trying to put mysql dump file on s3 bucket. It gives an error like ; sudo ./backup.sh WARNING: Module python-magic is not available. Guessing MIME types based on file extensions. ...
0
votes
0answers
418 views

How can I move s3 objects >5gb from one bucket to another?

Objects larger than 5GB are supported with multi-part uploading. We have 5gb+ files in one bucket already that I'd like to move to another, under the same AWS account. When I issue the command using ...
0
votes
0answers
199 views

Getting a simple list of changed files from unison

I have a filesystem that is changed on two servers and also needs to be replicated to Amazon S3. Until recently, syncing the filesystem between the two servers using Unison, and then copying to S3 ...
0
votes
0answers
81 views

Amazon S3 - Storage Class and Server Side Encryption

Ahhh! I am using Amazon S3 for some low price storage to clear down out SAN. I created the bucket and created a root folder. I set the storage class to standard and server side encryption AES. I ...
0
votes
0answers
133 views

Backing up files from my EC2 instance to S3

Hi I have a set of neat scripts that backup my dbs into db.sql.gz files under a folder on on ec2 host. My files are of the form (as you can see they are weekly rolling backups) ...
0
votes
0answers
155 views

s3fs not maintaining uid/gid

I'm mounting s3 using s3fs with the following command: sudo /usr/local/bin/s3fs -o use_cache=/tmp,uid=1000,gid=1000,allow_other svn.domain.com /svn Allow_other is confirmed to be working along with ...