This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

2
votes
2answers
159 views

Uploading to Amazon S3 from web link

I have huge amount of files on my old host... I would like to move all files to Amazon S3 since it seems very cheap. My internet connection at home is very slow, so I can't download them and upload it ...
2
votes
1answer
1k views

Best way to mirror data across two-node apache cluster

I'm architecting a new server structure and I plan to have two apache workhorses and a sql database behind both of them. I was wondering what the best way to mirror the data between the apache servers ...
2
votes
2answers
1k views

Best AWS Architecture?

The goal is to have as much redundancy as possible so that we can quickly recover from instance failures or any other issues. Our current AWS setup for a Rails WebApp that we run is as follows: EC2 ...
2
votes
3answers
1k views

Does cloud storage replicate the data over many datacenters if so it means i benefit content delivery network?

Let's assume that i want to use cloud storage service from one of the cloud storage provider, i got X gb structured and unstructured data and i will use this data as my contents of my interactive web ...
2
votes
1answer
228 views

Rsync to AWS S3 bucket

For a server I am hosting a website on I want to backup the data and settings to an S3 bucket. I found out that you can't directly use rsync to backup to an S3 bucket. Is there another way to achieve ...
2
votes
1answer
916 views

Amazon S3 - private file is still downloadable for everyone?

I have some secret files which I want to restrict anonymous user from viewing/downloading. I have tried to make it private by running something like: s3cmd setacl --acl-private s3://bucket/some/path/*...
2
votes
2answers
2k views

Anonymous access to S3 bucket only from my EC2 instances

Is there a way to let anonymous access to a certain S3 bucket only from my EC2 instances (all of them) within a single AWS account? I know I can use IAM roles, but I found it's just too many moving ...
2
votes
4answers
4k views

CNAME not resolving with nslookup, but dig works

I am trying to point a CNAME to a S3 bucket, but keep having problems with the CNAME not being found by nslookup (and by my browser, for that matter). When I use dig however, it shows me the correct ...
2
votes
3answers
824 views

Hosting a site on Amazon S3, URL keeps redirecting

I am trying to use Amazon S3 to host my webiste. I have the domain registered though namecheap. Whenever I navigate to mywebsite.com the URL in the browser changes to http://mywebsite.com.s3-website-...
2
votes
2answers
3k views

Create an Amazon EC2 instance with automatic access to private S3

I was wondering. Say I have a file in a private S3 bucket, and I then deploy an EC2 instance which has something like that in its rc.local: GET http://privateamazons3url.com How can I make it so ...
2
votes
1answer
1k views

s3cmd put only files that do not exist in destination bucket

I've been using s3cmd to sync local files on our server with the S3 bucket. There are over 200,000 files on the local server. Can't seem to find any documentation on the site that explains how to ...
2
votes
1answer
193 views

Restrict audio file access by domain on S3 and/or Apache

We has some proprietary audio files we are streaming via html5. These files sit in an Amazon S3 bucket. The audio files have special high and low pass filters and other scientific alterations and ...
2
votes
2answers
423 views

S3 upload delayed

I have 2 servers using the same data located in AWS S3 storage. One is processing HTTP traffic, the other does longer lived jobs. How it works is that HTTP one saves data to S3, dispatches event to ...
2
votes
2answers
599 views

AWS S3 and EBS backup?

I'm rather new to Amazon Web Services. I have a question regarding S3 and EBS. Do I have to backup an S3 or EBS service? Are these types of storage secure and backuped by amazon itself?
2
votes
3answers
1k views

uploading to amazon s3 using lftp

Is it possible to upload files from an ftp to an amazon s3 bucket using lftp? I tend to run this from an ec2 instance.
2
votes
2answers
112 views

Amazon Lambda for video and image processing?

The service I am building the architecture for has a video processing part in it. Basically, some people can upload videos (stored in S3) that will be split into frames with high resolution. From ...
2
votes
2answers
1k views

Is there a way to retrieve all contents from a versioned Amazon S3 bucket as of a particular datetime

Is it possible via the AWS CLI or via some third-party tool to easily download all contents of a versioned bucket as it was on a particular datetime? I.E. I would like to retrieve the version of each ...
2
votes
3answers
1k views

Should I use ec2 as a file server?

I need to be able to share user uploaded content across multiple EC2 application servers. I have looked at rsync, mounted NFS, and S3 as potential options of being able to share this data in almost ...
2
votes
1answer
252 views

How do I implement IP-based export restrictions for my data stored on Amazon S3?

A little background: Corporate wants to make our downloads unavailable to U.S. export-restricted countries like North Korea, Iran, Syria, etc. Restriction by IP-block is considered good enough. (Yes, ...
2
votes
2answers
5k views

How to calculate Amazon S3 prices

I am trying to estimate how much I need to pay to store images for my customers on Amazon S3. I use their calculator but the total seems to cheap. Feels like I am missing something. Each image is ...
2
votes
1answer
504 views

Change Amazon S3 storage class on A LOT of files

We are using Amazon S3 and a great deal of our files have the STANDARD storage class. We want to change this to REDUCED_REDUNDANCY class to save the extra money. However, is has been hard to find a ...
2
votes
2answers
595 views

Best way to implement Amazon CloudFront for established website

Some of you may have a solution for this, and I need to hear it. Basically, you have a website, a lot of categories ( games, blog etc ). The idea would be to use Amazon CDN for images, flv's, ...
2
votes
4answers
2k views

Whats the best cloud backup solution for a small scale server environment? [closed]

I have a server that runs a postgres database that contains about 200MB of data. Currently I have a cron job setup on my home computer which: ssh's into my server runs a remote script which makes a ...
2
votes
4answers
1k views

Full MySQL BAckup strategy

Here is our MySQL settings : 3 MySQL servers in a "replication-ring" : they're all Master and S1 is slave to S2, which is slave to S3 which is slave to S4. Up until now we would do snapshots of the ...
2
votes
1answer
339 views

Can you have private URLs in Amazon's CloudFront?

In Amazon's S3 storage system you can have sort of private URLs using a special query string and expiring the link after a while. Is such a private system possible using Amazon's CloudFront, which is ...
2
votes
1answer
63 views

How to secure the db dump taken from daily run script

In one of my mysql slave servers I have written a daily run script, which 1) stops slave, 2) takes a db dump, 3) starts slave again, 4) encrypts it, 5) copies it to my s3-bucket. I am using aws-cli ...
2
votes
1answer
156 views

Does the AWS CLI mv command also do a copy-and-delete?

I am investigating renaming bundles in AWS S3. I would prefer to use the SDK, but it appears that the SDK only supports Copy and Delete. This because the object keys are used for partitoning the ...
2
votes
1answer
2k views

Url rewriting in Amazon S3

I have a webapp hosted in Amazon S3, the webapp is build with jQuery Mobile and I use changePage lo navigate between pages, I'm not using the hash operator(#) when I change the page. I'm having ...
2
votes
1answer
2k views

using CloudFormation with an existing S3 bucket

Using CloudFormation, I want to set some of the properties in AWS::S3::Bucket on an existing bucket. In other words, I don't want to create the bucket, I just want to enforce some of the settings. ...
2
votes
1answer
1k views

What is your general strategy to backup S3 buckets?

We are starting a project which involves storing large data quantities in S3. S3 is good at scaling and we've expect to have up to 5TB and million of files in bucket. While i may trust Amazon in ...
2
votes
2answers
8k views

How to config Amazon Route53 working without www in sub-domain

edit: Amazon now supports this. http://aws.typepad.com/aws/2012/12/root-domain-website-hosting-for-amazon-s3.html I have my domain config in Route53 at Amazon AWS exec ping command in my domain ...
2
votes
3answers
474 views

What is the best way to backup mysql in s3?

mysqldump is probably not the best idea on running backups every x hours but is the one that we are currently using. The backups are around 150 Megs each so sending it to other machines could waste ...
2
votes
1answer
228 views

How can I turn an existing Windows Server 2008 EBS-backed instance into a S3-backed (instance-backed) AMI?

I understand there is no direct conversion to get from EBS to S3. But is there some series of steps that can make this happen, or is rebuilding the image from scratch the best option? Is there a way ...
2
votes
1answer
734 views

S3 to cloudfront edge location vs cloudfront to end user transfer ratio

We host videos in amazon S3 and use cloudfront to deliver them. Here is last months numbers: 4,2 TB total traffic from cloudfront to end user. 4 TB of this is in europe, the rest mainly in the US. ...
2
votes
1answer
3k views

Cannot upload files bigger than 8GB to Amazon S3 by multi-part upload due to broken pipe

I implemented S3 multi-part upload, both high level and low level version, based on the sample code from http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?HLuploadFileJava.html and http:...
2
votes
1answer
898 views

Alias multiple DNS entries to one Amazon S3 Bucket

I have a bucket on Amazon S3. Lets call it "webstatic.mydomain.com". I have a DNS alias setup for that bucket webstatic.mydomain.com CNAME -> web-static.mydomain.com.s3.amazonaws.com. This all works ...
2
votes
2answers
1k views

Understanding Amazon AWS usage data

I love Amazon AWS, but I'm tearing my hair out over the obscure usage data they give you. All they seem to have is either an extremely high level summary that just shows you your total monthly cost ...
2
votes
3answers
52 views

how does AWS S3 bucket Redundant storage works with filepicker/filestack

I am using Filestack https://www.filestack.com/ as fileserver and Amazon S3 as filestorage as complete storage solution for my clients. Filestack offers CDN (Content Delivery Network) which serves ...
2
votes
1answer
289 views

How to upload a large file using aws commandline when connection may be unreliable?

I've been having trouble uploading a large (800MB) file to s3, using the aws commandline tool. The first attempt completed (after many hours) but was not visible, and I was advised (here) that it has ...
2
votes
1answer
2k views

HTTPS access to S3 bucket through Cloudflare

I'm trying to use Cloudflare to provide https access to an s3 bucket. I've setup the CNAME entry in Cloudfront with pointing to the bucket and I've verified I can access it via http but when I try and ...
2
votes
1answer
413 views

s3 policy for quota

Is there a s3 policy rule that can only allow a user to upload so many GB of files? Basically looking to set a quota on the objects a user can upload. Here is the rule set that I have now: { "...
2
votes
1answer
3k views

How to copy all s3 buckets to another AWS account?

I'm copying all EC2 instances to another AWS account (creating the AMIs, sharing them with that account and running the instances from AMIs there). I also need to move my S3 buckets since the ...
2
votes
1answer
1k views

Best failover strategy for e-mail servers on AWS to ensure high availability

We have our e-mail server hosted on AWS. Last week Amazon had a failure in their East Coast region which brought down our server along with many others. We now want to implement a failover strategy ...
2
votes
1answer
112 views

Performance issue in using Amazon S3 for run-time updates

I have a web application hosted in Amazon EC2 as well as in Rackspace. Users can upload files through this application and also modify them. Those files are stored in local storage in each of these ...
2
votes
1answer
1k views

Use Microsoft DPM 2010 with Azure or Amazon S3?

Does anyone know of any software that hooks up with Microsoft's DPM 2010 or even the new 2012 to ship data to cloud storage providers like Azure or S3? We utilize D2D backups today but want a more ...
2
votes
1answer
753 views

Hosting animated GIF banners on S3

I have uploaded some animated GIF banners to our S3 account and when we use the web url provided by S3 and try to view the banners in a browser, instead of display the banners in the browsers, I get ...
2
votes
3answers
364 views

Bandwidth-friendly backup strategy using tar?

We've used tar to backup and compress (gzip) selected directories on our file server with very good results until recently. Each and every one of our backups are stored on mirrored (RAID) harddrives ...
2
votes
1answer
239 views

Amazon S3 Set Expires header

Is there any way to tell Amazon S3 that all existing and future files in some bucket should have some speicific Expires header?
2
votes
1answer
495 views

Simple command line storage tools for Azure Storage (like s3cmd, sdbShell)?

I'm currently backing up/migrating Amazon S3 and SimpleDB data from AWS to my on premise systems using sdbShell and s3cmd. Works great. Now we're putting an app up on Microsoft Azure and I want to ...
2
votes
1answer
144 views

Easiest way to switch many files to RRS in Amazon S3?

I have an S3 bucket with many thousands of files copied from a linux machine. I would like to switch all files within a folder to RRS (Reduced Redundancy Storage). How can I do this without deleting ...