This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

0
votes
1answer
335 views

AWS instances: AMI bundles and snapshots and bears, oh my

I've googled a good amount and am still pretty confused and feeling like I'm missing or overlooking something. My goal is to restore a host using a bundle. I have an AMI bundle of a now-nonexistent ...
2
votes
1answer
450 views

Amazon S3 Website & MySQL Backup

I have a website (a digital asset management system/gallery - http://www.resourcespace.org) that has a huge amount of images. The total size of the website, including the images is approximately 6gb. ...
0
votes
1answer
89 views

apache sitemap rewrite to s3

I wanted to submit sitemap to google webmaster, google state only accept sitemap from my site Option 1 - simple rewrite I know this already working: RewriteRule /sitemap.txt http://S3_URL [L] ...
2
votes
2answers
195 views

Amazon CloudFrount on EC2

I know that i can mount s3 buckets with s3fs on EC2 instance. So i have next question can i somehow mount Amazon Cloud Front (my S3 buckets files) in EC2 instance?
4
votes
2answers
2k views

Snapshots == EBS backed instance, persistence, and S3 storage?

I want to clear up some confusion and clarify understanding on EC2 instances being EBS backed. If you start an AMI from a community image, configure your server as you want it configured, then use ...
2
votes
2answers
449 views

CloudFront - Users requests static files non-stop to make your bill high?

I had this weird wondering once. If I am using CloudFront as my CDN, I pay per requests made on my static files. So here comes the wondering. Can one keep requesting the same file in a non-stop ...
1
vote
2answers
2k views

How to calculate Amazon S3 prices

I am trying to estimate how much I need to pay to store images for my customers on Amazon S3. I use their calculator but the total seems to cheap. Feels like I am missing something. Each image is ...
4
votes
2answers
1k views

What design features make Joyent's ZFS and Amazon's EBS (S3) reliable?

I know this isn't exactly an apples to apples comparison, what I'm trying to evaluate is which one is safer - less likely to lose data. Joyent's SmartOS uses ZFS to store data whereas an EC2 machine ...
2
votes
1answer
620 views

Use Microsoft DPM 2010 with Azure or Amazon S3?

Does anyone know of any software that hooks up with Microsoft's DPM 2010 or even the new 2012 to ship data to cloud storage providers like Azure or S3? We utilize D2D backups today but want a more ...
1
vote
2answers
158 views

How to show a “linked from site X” placeholder instead of actual photo?

I’m hosting a bunch of photos on Amazon S3 and also running Amazon CloudFront for them. Also, I’m running a custom domain name. Now, in case someone tries to embed the image to another website, I’d ...
2
votes
1answer
398 views

Using an IP Address to Point to Amazon S3 Storage

Is there a way to get an IP address to refer to Amazon S3 Storage location? The current solution my company is thinking of going with involves setting up a EC2 server with an Elastic IP and using ...
2
votes
1answer
530 views

S3 to cloudfront edge location vs cloudfront to end user transfer ratio

We host videos in amazon S3 and use cloudfront to deliver them. Here is last months numbers: 4,2 TB total traffic from cloudfront to end user. 4 TB of this is in europe, the rest mainly in the US. ...
-1
votes
2answers
169 views

Cross platform, blob storage system

I have a couple of different servers around the house, and a lot of storage on each of these machines. I also have a couple of dedicated servers, and some VMs on these boxes... Most of the boxes and ...
0
votes
1answer
102 views

How do I adjust this python script to backup to Amazon S3

The following script is nearly perfect for my backup needs: http://pastie.org/3368661 However, it only allows you to backup to the server or a remote FTP. I would like to add an option to backup to ...
10
votes
3answers
3k views

Any need to make backup of data on Amazon S3?

I'm hosting 200 GB of product images at S3 (this is my primary file host). Do I need to back that data up somewhere else, or is S3 safe as it is? I have been experimenting with mounting the S3 ...
0
votes
1answer
362 views

A uploaded file is 0 byte on Amazon S3 using django

I've symbolic linked a S3 directory on my EC2 server. But a uploaded file was 0 byte when I uploaded file. It was work well before (I used just web server directory) I don't have any idea dealing ...
3
votes
2answers
549 views

Tracking down “Connection reset” errors in Linux

I'm handling large numbers of concurrent downloads (approx. 500 per server) using Java. All the files are being downloaded from Amazon S3, and the downloading server is an EC2 m1.large instance. ...
7
votes
2answers
7k views

Is mounting S3 buckets directly into EC2 instances safe?

as the title of my question says, I would like to know whether this is a good practice or not, I know I can mount it in an EC2 instance but I was wondering if, as I will be adding more instances to ...
2
votes
3answers
931 views

projects or packages to emulate a tape drive with a Amazon S3 or OpenStack Swift compatible backend?

I've got some data that is being backed up to tape via bacula, and at some point soon I am going to need to migrate to non-tape storage. Bacula supports to-disk archiving, but it occurred to me that ...
0
votes
1answer
613 views

Does anyone have a config to process AWS CloudFront/S3 logs with AWStats (or similar sw)?

I don't want to give an external company like s3stat access to my logs. I know that AWS logs S3 and Cloud Front in a format readable by AWStats. Has anyone used AWSats to analyze them? S3stat used ...
-1
votes
1answer
302 views

Which of snapshots I can safely remove?

If you make periodic snapshots of a volume, the snapshots are incremental so that only the blocks on the device that have changed since your last snapshot are incrementally saved in the new ...
19
votes
4answers
13k views

How to move files between two S3 buckets with minimum cost?

I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone. How could I ...
0
votes
1answer
162 views

Is uploading a 500mb file to s3 an expensive operation? Any way to limit cpu usage etc?

On a production server, I'm planning on uploading a single file or a set of files that may be 500 megs in size. I havent' test it just yet, but just want some opinions on whether you think this will ...
5
votes
3answers
3k views

Forcing CloudFront to pass-through the latest HTML file from S3

Background I'm hosting a static site on S3, with CloudFront over the top. The issue I have is with my HTML files. According to CloudFront's FAQ: Amazon CloudFront uses these cache control ...
2
votes
2answers
1k views

Amazon EC2 - What's the point of having Instance storage if it is non-permanent and EBS provides a permanent store?

I'm specifically looking at the Standard Large On-Demand instance which has 850 GB of instance storage. What I really don't understand is why have 850 GB of storage on the instance if that data will ...
-1
votes
3answers
323 views

amazon s3 vs amazon ec2 regarding cost, and which better for image hosting website? [closed]

I want to know which will cost more, s3 or ec2. I need to know by example to estimate my future payments. Also which of them will be better for an image hosting website?
4
votes
3answers
3k views

Uploading 200GB of files to S3

What's the best way to upload 200GB tar.gz files to S3 in Linux? On researching I found that S3 limit on objects has been increased to 5TB and came to know about the multi part upload mechanism to ...
3
votes
1answer
286 views

Using amazon S3 as storage for attachments in a web-mail system

I am looking to use S3 as data storage service for a mail system. The idea is to allow users to upload files that serve as attachments to their emails, which are then available to download from the ...
1
vote
1answer
463 views

Protected bucket @ S3 - Access Denied

I'm trying to grant RW access to a given bucket to a specific user using the following bucket policy: { "Id": "Policy1322043790167", "Statement": [ { "Sid": "Stmt9999043784080", ...
2
votes
0answers
259 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
6
votes
5answers
6k views

s3cmd fails too many times

It used to be my favorite backup transport agent but now I frequently get this result from s3cmd on the very same Ubuntu server/network: root@server:/home/backups# s3cmd put bkup.tgz s3://mybucket/ ...
1
vote
1answer
197 views

How to configure S3 or DNS to handle incomplete name (sans www) for web site?

I have a set up a bucket called "www.mydomainname.com" to host my website and I have configured the CNAME such that "www.mydomainname.com" points to the my endopint ...
0
votes
1answer
98 views

Running my own CDN alongside Amazons S3 for nearby visitors

I am running a blog on shared webhosting. The website itself is on the hosting, but my files right now are on Amazons S3 CDN. I have made a bucket lockated in the US, and I have setup a virtual host ...
1
vote
2answers
75 views

Cloud storage options for client downloads

I'm looking for a way to distribute sample files to clients. Requirement is that it's (relatively) simple to upload files, a link can be sent to a client who can download the file with a browser, the ...
3
votes
1answer
917 views

Backing up AFP Shares from my FreeNAS server

I have chosen to use Amazon S3 to back up my AFP shares from my FreeNAS server. What is the best / most efficient (automated) way in which to get my data from the AFP shares to Amazon S3. I have ...
1
vote
4answers
326 views

Best practice for synchronizing sources across servers (Amazon EC2)

I'm moving from single app server to load balanced config. In the single server config I synchronized the sources by S3 cmd sync s3 every few minutes and on hostup (Scalr event) I sync the sources ...
8
votes
2answers
4k views

EC2 - Shared Storage - S3FS or EBS?

Building my web service onto EC2 right now and have a single instance behind a load balancer. I will of course cater for multiple instances. My initial idea was to run all the instances a dumb ...
1
vote
1answer
903 views

Mounting an S3 bucket onto a AWS Ubuntu instance issues

i'm using s3fuse to mount an s3 bucket onto ubuntu, so that i can take advantage of the storage space. here are some environment info: System Info: Linux ip-xxx-xxx-xxx-xxx 2.6.38-8-virtual ...
0
votes
1answer
675 views

Writing to an s3 mounted directory

hopefully someone can advise. I've recently mounted an s3 bucket to a directory /s3 on my ubuntu amazon ec2 instance -following: http://code.google.com/p/s3fs/wiki/FuseOverAmazon. This was set up in ...
0
votes
1answer
184 views

How can one write a custom certificate verification logic on Amazon S3? [closed]

From the developer guide: When using virtual hosted-style buckets with SSL, the SSL wild card certificate only matches buckets that do not contain periods. To work around this, use HTTP or ...
2
votes
1answer
248 views

Hosting animated GIF banners on S3

I have uploaded some animated GIF banners to our S3 account and when we use the web url provided by S3 and try to view the banners in a browser, instead of display the banners in the browsers, I get ...
0
votes
1answer
161 views

Alternative storage for SVN backup with rsync

I've been backing up my SVN repo and trac environment using a weekly cron job and 2 custom scripts. This has been going to Joyent Strongspace; but since they have discontinued the service, I need an ...
1
vote
1answer
115 views

Restricting console permissions in IAM for S3

I want to be able to allow users created through IAM to be able to view one bucket in the management console. Furthermore, I want to restrict it to a folder within the bucket, such that the ...
1
vote
1answer
118 views

Cname and Amazon S3 failure

I had a quick question. If you set up a cname for your amazon bucket and lets say your main amazon s3 bucket goes down, does cname you created serve files, in other words, do your files get redirected ...
3
votes
1answer
125 views

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which ...
2
votes
2answers
701 views

How can I secure files on S3?

I want to control who can download a file via an app (as if it was held on the local FS). What's the right approach for doing this, considering S3 can't be mounted as a POSIX compliant file system?
0
votes
2answers
19 views

S3 upload feasibility

I've already asked this question on the AWS forums but haven't had a response yet so I thought I'd up it up a bit to see if anyone here might have an idea about this: We've a client who's talking ...
0
votes
2answers
383 views

Best practice for storing read-only instance data?

So, i'm currently writing my bachelor thesis and my job is to cloudify a webservice which recommends a soundtrack for images. The main part of the recommendation process is a lookup on a ~40gb index ...
2
votes
2answers
112 views

Uploading to Amazon S3 from web link

I have huge amount of files on my old host... I would like to move all files to Amazon S3 since it seems very cheap. My internet connection at home is very slow, so I can't download them and upload it ...
1
vote
1answer
206 views

Change Amazon S3 storage class on A LOT of files

We are using Amazon S3 and a great deal of our files have the STANDARD storage class. We want to change this to REDUCED_REDUNDANCY class to save the extra money. However, is has been hard to find a ...