This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

3
votes
2answers
205 views

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which ...
1
vote
2answers
321 views

HTTPS access to S3 bucket through Cloudflare

I'm trying to use Cloudflare to provide https access to an s3 bucket. I've setup the CNAME entry in Cloudfront with pointing to the bucket and I've verified I can access it via http but when I try and ...
1
vote
2answers
386 views

Taking Cassandra Backups

We currently have 12 nodes running in our Cassandra cluster. Ultimately even if a couple of the nodes go down, we're still up and running. The paranoia in me would like to do at least one backup a day ...
1
vote
2answers
115 views

How to list s3fs mounts?

How can I list the s3fs mounts that exist on an ubuntu system? I'd like to know to which bucket each mount is mapped. Specifically, I have a specific mount (e.g. ~/s3/mymount), and would like to know ...
1
vote
2answers
332 views

How to deploy local project to Amazon

I have a small webapp written in Python/Django which works fine on my local machine. I've been tinkering and setting up my server on the free tier of Amazon EC2 by following online tutorials. However, ...
1
vote
2answers
976 views

s3fs Input/output error

I'm trying to set up a backup system with s3fs and the Amazon S3 service. I followed this this guide and this guide. Tailing /var/log/messages I get: Aug 28 13:37:46 server s3fs:###response=403 I ...
0
votes
2answers
136 views

Looking for a high IO s3 “compatible” storage solution

Help me serverfault! I am looking for a high IO (2 million mostly put transactions per day of 7-10k xml chunks) storage solution that is amazon s3 compatible that I can house in my datacenter. I am ...
0
votes
2answers
849 views

Enable Cross Origin Scripting on Nginx from retina.js to load files from another URL?

Im running a retina image plugin along with W3 Cache for CDN support. W3 cache moves images to S3, but the JS file won't server the files from there due to the difference in URL (it goes from h ...
0
votes
2answers
25 views

S3 upload feasibility

I've already asked this question on the AWS forums but haven't had a response yet so I thought I'd up it up a bit to see if anyone here might have an idea about this: We've a client who's talking ...
1
vote
1answer
359 views

Reducing latency for different geographic regions on Amazon Cloud

I have got an application which has three components Application code : Amazon EC2 US-EAST-1 instance Application images, and other static data : Amazon S3 with CloudFront Application Database : ...
0
votes
1answer
15 views

AWS S3 Sync options duplicates / weird behaviour

I am trying to sync a folder with 2M+ files to S3, everything went OK but then 40,000 files were not uploaded (server crashed randomly), when I tried to do a sync command again, it started from 0, ...
0
votes
1answer
65 views

Where are AMIs stored?

It seems like an very simple question, but yet I could not find the answer. When I make an instance on Amazon AWS and then make some changes and create an AMI where is that AMI stored? S3? Will making ...
0
votes
1answer
324 views

Setting cache for already uploaded amazon s3 files

In amazon s3 for our site through s3cmd tool we uploaded around 60GB of files including mp3, jpg to a bucket. But now when we checked site slowness files are not caching and further checking we came ...
0
votes
1answer
116 views

AWSCLI s3 sync doesn't create directory

Hi try to sync a local directory to a S3 bucket. The files in the directory get synced perfectly, just the directory isn't created at all. Can you guys give me a hint how to do it? "newdir" contains ...
0
votes
1answer
175 views

Duplicity Backups (via Duply) to S3 - high server load

We have been doing incremental backups using Duply on our main server, to an S3 bucket. However, we've found that there is a high server load during the backups (its an Amazon EC2 server). We're ...
0
votes
1answer
192 views

rebuild yum index on aws s3

I am trying to rebuild yum repo on aws S3 after adding new packages. Here are few commands I am trying, but it is not helping. [root@chucks ~]$ createrepo --baseurl ...
0
votes
1answer
805 views

Amazon Cloudfront with S3 Redirect

I'm trying to do something that should be relatively straight-forward - I want to setup a few domains and subdomains to redirect to the core domain for our site, but I want to put the redirects out on ...
0
votes
1answer
233 views

can't get s3 authentication to work to load data into redshift

I'm working on getting data into a sample Redshift cluster for a client, but I can't get authenticated to S3. There's been a recent change so that my own 'root user' access keys aren't available for ...
0
votes
1answer
807 views

s3cmd: Deleting files or folder does not work

s3cmd s3://MYBUCKET/FOLDER does not work. (folder is empty) s3cmd s3://MYBUCKET/FILE does not work. Output of both commands is "File FILE sucessfully deleted", yet the file remains on the bucket. ...
0
votes
1answer
74 views

Glacier retrieval notification of s3 objects

I have some s3 buckets which send their objects to glacier after x days of creation. When I go to retrieve those objects on the s3 interface, I need to keep reloading page to see when restoration has ...
0
votes
1answer
440 views

Move files from ftp server to s3

I would like to set up an ftp server, where users will upload files, and for each file, put it on s3 storage, and delete it from the ftp server. (the server runs on ec2 ubuntu) Here are the stuff I ...
0
votes
1answer
146 views

AWS route53 forwarding

Essentially, I need to know how to point one URL/folder to a web server and a different URL/folder to an EC2 app server. I'm trying to setup a SAAS with the following URL configuration. ...
0
votes
1answer
142 views

Can AWS S3 files in buckets get deleted by a download timing out?

A few AWS S3 files might have gotten deleted while trying to download everything for a local backup. The download/connection timed-out a couple times. The next day several HTML/JS files are missing ...
0
votes
1answer
229 views

Amazon S3 map url to backend server like Apache's <Location /api> ProxyPass

Is there a way to map a URL to a backend server in S3. I am not looking for redirects, but something on the lines of Apache's Location directive with ProxyPass. ...
0
votes
1answer
592 views

AWS EC2 Mailserver Failover Strategies done right

I'm researching in this topic really hard the last few days and i just want to discuss this with a few specific questions - i did not find any suitable thread here that is covering my needs and ...
0
votes
1answer
558 views

Unable to read S3 file from interactive pig job flow

I'm unable to read a simple test file on S3 from an interactive pig job flow (hadoop, elastic map reduce), and I'm not sure why. I have two S3 buckets. Let's call them unmounted_bucket, and ...
0
votes
1answer
119 views

autoscale static content on ec2 using local cache

Hello Serverfault members. I want to achieve the following setup on EC2: A load balancer starts and stops instances which only serve static content. (a javascript based application) I do not want to ...
0
votes
1answer
392 views

Amazon AWS Store Wordpress uploads on S3

I'm getting ready to move my company's website and order management application to AWS. We use wordpress as the 'face' of our company. Our order management app is for internal use. I'm using Elastic ...
0
votes
1answer
366 views

Upload database backup from mysql to Amazon S3 or Glacier without creating local file

Is there a tool that makes possible to backup a Mysql database to Amazon S3 or Amazon Glacier without having o create a local file with the database contents? Something like that: mysqldump -u ...
0
votes
1answer
187 views

configure s3fs mount for write-only

I am using s3fs to mount a S3 buckets on an Ubunutu server to store my daily backups there. Considering security this would allow a potential hacker to delete all my backups from the mounted bucket. ...
0
votes
1answer
431 views

How to find out what's missing from my iptables, why it's blocking my AWS transfer?

I use ManageWP to backup WordPress on my VPS then upload that to Amazon S3. It's failing with "Error 503 Service Unavailable" at the upload part when I have iptables running. When the iptables ...
0
votes
1answer
303 views

Amazon S3 - can't upload files into the same folder using s3sync.exe

I'm trying to upload a file using S3sync.exe into a certain folder in my S3 bucket. This is the command line : S3Sync.exe -AWSAccessKeyId accKey -AWSSecretAccessKey secKey -RequestEndpoint ...
0
votes
1answer
165 views

s3 multiple write issue

I've built an application which forks process pools to fetch web content. I have 3 ec2 instances mounted onto one s3 bucket via fuse. I'm experiencing intermittent errors writing files. From what I ...
0
votes
1answer
161 views

S3 backup: bucket vs. volume

I've inherited a project that has been pushing encrypted (GPG) DB backups via a backup script to an S3 bucket. That's all been well and good, but now that bootable EBS instances (where the project ...
0
votes
1answer
541 views

S3sync not working

I managed to get s3sync to upload my test folder to Amazon S3 and can see it in the MWS Managment Console. Downloading the data back to a test folder results in the following error message: ...
0
votes
1answer
68 views

Backing Up User Data when data is not in use. Should I be concerned?

This may be a dumb question. I would like to use duplicity to make backups to Amazon S3 of directories, each of which contains a different user's data. Each directory could be written to at any time. ...
0
votes
1answer
419 views

CDN Rerouting on 404 (file not yet in synch with original storage)

Here is the problem. I've setup my app(on EC2) to store uploaded images directly on Amazon S3. I'd like to be able to serve static files(cdn) from my 'home' server so I wrote script that does sync ...
-1
votes
1answer
360 views

AWS s3 - How to serve different file based on url parameter

For legacy reasons I need to serve different files from amazon s3 based on query string parameters. For example myfile.php?type=1 should serve other file than myfile.php?type=2. the files themselves ...