This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

3
votes
2answers
192 views

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which ...
2
votes
2answers
224 views

Amazon AWS EC2 instance Exported to local machine

I have several Amazon AWS EC2 virtual machines that I created on AWS EC2. They were not imported, they were created there. I would like to export these machines in order to achieve local backups, ...
1
vote
2answers
198 views

HTTPS access to S3 bucket through Cloudflare

I'm trying to use Cloudflare to provide https access to an s3 bucket. I've setup the CNAME entry in Cloudfront with pointing to the bucket and I've verified I can access it via http but when I try and ...
1
vote
2answers
363 views

Taking Cassandra Backups

We currently have 12 nodes running in our Cassandra cluster. Ultimately even if a couple of the nodes go down, we're still up and running. The paranoia in me would like to do at least one backup a day ...
1
vote
2answers
115 views

How to list s3fs mounts?

How can I list the s3fs mounts that exist on an ubuntu system? I'd like to know to which bucket each mount is mapped. Specifically, I have a specific mount (e.g. ~/s3/mymount), and would like to know ...
1
vote
2answers
326 views

How to deploy local project to Amazon

I have a small webapp written in Python/Django which works fine on my local machine. I've been tinkering and setting up my server on the free tier of Amazon EC2 by following online tutorials. However, ...
1
vote
2answers
951 views

s3fs Input/output error

I'm trying to set up a backup system with s3fs and the Amazon S3 service. I followed this this guide and this guide. Tailing /var/log/messages I get: Aug 28 13:37:46 server s3fs:###response=403 I ...
0
votes
2answers
133 views

Looking for a high IO s3 “compatible” storage solution

Help me serverfault! I am looking for a high IO (2 million mostly put transactions per day of 7-10k xml chunks) storage solution that is amazon s3 compatible that I can house in my datacenter. I am ...
0
votes
2answers
794 views

Enable Cross Origin Scripting on Nginx from retina.js to load files from another URL?

Im running a retina image plugin along with W3 Cache for CDN support. W3 cache moves images to S3, but the JS file won't server the files from there due to the difference in URL (it goes from h ...
0
votes
2answers
24 views

S3 upload feasibility

I've already asked this question on the AWS forums but haven't had a response yet so I thought I'd up it up a bit to see if anyone here might have an idea about this: We've a client who's talking ...
0
votes
1answer
139 views

Duplicity Backups (via Duply) to S3 - high server load

We have been doing incremental backups using Duply on our main server, to an S3 bucket. However, we've found that there is a high server load during the backups (its an Amazon EC2 server). We're ...
0
votes
1answer
166 views

rebuild yum index on aws s3

I am trying to rebuild yum repo on aws S3 after adding new packages. Here are few commands I am trying, but it is not helping. [root@chucks ~]$ createrepo --baseurl ...
0
votes
1answer
667 views

Amazon Cloudfront with S3 Redirect

I'm trying to do something that should be relatively straight-forward - I want to setup a few domains and subdomains to redirect to the core domain for our site, but I want to put the redirects out on ...
0
votes
1answer
214 views

can't get s3 authentication to work to load data into redshift

I'm working on getting data into a sample Redshift cluster for a client, but I can't get authenticated to S3. There's been a recent change so that my own 'root user' access keys aren't available for ...
0
votes
1answer
614 views

s3cmd: Deleting files or folder does not work

s3cmd s3://MYBUCKET/FOLDER does not work. (folder is empty) s3cmd s3://MYBUCKET/FILE does not work. Output of both commands is "File FILE sucessfully deleted", yet the file remains on the bucket. ...
0
votes
1answer
71 views

Glacier retrieval notification of s3 objects

I have some s3 buckets which send their objects to glacier after x days of creation. When I go to retrieve those objects on the s3 interface, I need to keep reloading page to see when restoration has ...
0
votes
1answer
397 views

Move files from ftp server to s3

I would like to set up an ftp server, where users will upload files, and for each file, put it on s3 storage, and delete it from the ftp server. (the server runs on ec2 ubuntu) Here are the stuff I ...
0
votes
1answer
140 views

AWS route53 forwarding

Essentially, I need to know how to point one URL/folder to a web server and a different URL/folder to an EC2 app server. I'm trying to setup a SAAS with the following URL configuration. ...
0
votes
1answer
126 views

Can AWS S3 files in buckets get deleted by a download timing out?

A few AWS S3 files might have gotten deleted while trying to download everything for a local backup. The download/connection timed-out a couple times. The next day several HTML/JS files are missing ...
0
votes
1answer
223 views

Amazon S3 map url to backend server like Apache's <Location /api> ProxyPass

Is there a way to map a URL to a backend server in S3. I am not looking for redirects, but something on the lines of Apache's Location directive with ProxyPass. ...
0
votes
1answer
556 views

AWS EC2 Mailserver Failover Strategies done right

I'm researching in this topic really hard the last few days and i just want to discuss this with a few specific questions - i did not find any suitable thread here that is covering my needs and ...
0
votes
1answer
507 views

Unable to read S3 file from interactive pig job flow

I'm unable to read a simple test file on S3 from an interactive pig job flow (hadoop, elastic map reduce), and I'm not sure why. I have two S3 buckets. Let's call them unmounted_bucket, and ...
0
votes
1answer
116 views

autoscale static content on ec2 using local cache

Hello Serverfault members. I want to achieve the following setup on EC2: A load balancer starts and stops instances which only serve static content. (a javascript based application) I do not want to ...
0
votes
1answer
377 views

Amazon AWS Store Wordpress uploads on S3

I'm getting ready to move my company's website and order management application to AWS. We use wordpress as the 'face' of our company. Our order management app is for internal use. I'm using Elastic ...
0
votes
1answer
352 views

Upload database backup from mysql to Amazon S3 or Glacier without creating local file

Is there a tool that makes possible to backup a Mysql database to Amazon S3 or Amazon Glacier without having o create a local file with the database contents? Something like that: mysqldump -u ...
0
votes
1answer
183 views

configure s3fs mount for write-only

I am using s3fs to mount a S3 buckets on an Ubunutu server to store my daily backups there. Considering security this would allow a potential hacker to delete all my backups from the mounted bucket. ...
0
votes
1answer
422 views

How to find out what's missing from my iptables, why it's blocking my AWS transfer?

I use ManageWP to backup WordPress on my VPS then upload that to Amazon S3. It's failing with "Error 503 Service Unavailable" at the upload part when I have iptables running. When the iptables ...
0
votes
1answer
288 views

Amazon S3 - can't upload files into the same folder using s3sync.exe

I'm trying to upload a file using S3sync.exe into a certain folder in my S3 bucket. This is the command line : S3Sync.exe -AWSAccessKeyId accKey -AWSSecretAccessKey secKey -RequestEndpoint ...
0
votes
1answer
160 views

s3 multiple write issue

I've built an application which forks process pools to fetch web content. I have 3 ec2 instances mounted onto one s3 bucket via fuse. I'm experiencing intermittent errors writing files. From what I ...
0
votes
1answer
158 views

S3 backup: bucket vs. volume

I've inherited a project that has been pushing encrypted (GPG) DB backups via a backup script to an S3 bucket. That's all been well and good, but now that bootable EBS instances (where the project ...
0
votes
1answer
539 views

S3sync not working

I managed to get s3sync to upload my test folder to Amazon S3 and can see it in the MWS Managment Console. Downloading the data back to a test folder results in the following error message: ...
0
votes
1answer
68 views

Backing Up User Data when data is not in use. Should I be concerned?

This may be a dumb question. I would like to use duplicity to make backups to Amazon S3 of directories, each of which contains a different user's data. Each directory could be written to at any time. ...
0
votes
1answer
411 views

CDN Rerouting on 404 (file not yet in synch with original storage)

Here is the problem. I've setup my app(on EC2) to store uploaded images directly on Amazon S3. I'd like to be able to serve static files(cdn) from my 'home' server so I wrote script that does sync ...
-1
votes
1answer
355 views

AWS s3 - How to serve different file based on url parameter

For legacy reasons I need to serve different files from amazon s3 based on query string parameters. For example myfile.php?type=1 should serve other file than myfile.php?type=2. the files themselves ...