This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

4
votes
2answers
455 views

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which ...
1
vote
2answers
120 views

AWS S3 Sync options duplicates / weird behaviour

I am trying to sync a folder with 2M+ files to S3, everything went OK but then 40,000 files were not uploaded (server crashed randomly), when I tried to do a sync command again, it started from 0, ...
1
vote
2answers
260 views

How to list s3fs mounts?

How can I list the s3fs mounts that exist on an ubuntu system? I'd like to know to which bucket each mount is mapped. Specifically, I have a specific mount (e.g. ~/s3/mymount), and would like to know ...
1
vote
2answers
362 views

How to deploy local project to Amazon

I have a small webapp written in Python/Django which works fine on my local machine. I've been tinkering and setting up my server on the free tier of Amazon EC2 by following online tutorials. However, ...
1
vote
2answers
1k views

s3fs Input/output error

I'm trying to set up a backup system with s3fs and the Amazon S3 service. I followed this this guide and this guide. Tailing /var/log/messages I get: Aug 28 13:37:46 server s3fs:###response=403 I ...
0
votes
2answers
52 views

CNAME to S3 returns the S3 URL

I've setup a static website on S3 under a www.mydomain.com bucket. I also created a CNAME record for it in Namecheap looking like this: Type: CNAME Record Host: www Value: www.mydomain.com.s3-...
0
votes
2answers
1k views

Enable Cross Origin Scripting on Nginx from retina.js to load files from another URL?

Im running a retina image plugin along with W3 Cache for CDN support. W3 cache moves images to S3, but the JS file won't server the files from there due to the difference in URL (it goes from h ttp://...
0
votes
2answers
32 views

S3 upload feasibility

I've already asked this question on the AWS forums but haven't had a response yet so I thought I'd up it up a bit to see if anyone here might have an idea about this: We've a client who's talking ...
0
votes
1answer
303 views

Can AWS S3 files in buckets get deleted by a download timing out?

A few AWS S3 files might have gotten deleted while trying to download everything for a local backup. The download/connection timed-out a couple times. The next day several HTML/JS files are missing ...
0
votes
1answer
328 views

Amazon S3 map url to backend server like Apache's <Location /api> ProxyPass

Is there a way to map a URL to a backend server in S3. I am not looking for redirects, but something on the lines of Apache's Location directive with ProxyPass. http://httpd.apache.org/docs/current/...
0
votes
1answer
837 views

AWS EC2 Mailserver Failover Strategies done right

I'm researching in this topic really hard the last few days and i just want to discuss this with a few specific questions - i did not find any suitable thread here that is covering my needs and ...
0
votes
1answer
1k views

Unable to read S3 file from interactive pig job flow

I'm unable to read a simple test file on S3 from an interactive pig job flow (hadoop, elastic map reduce), and I'm not sure why. I have two S3 buckets. Let's call them unmounted_bucket, and ...
0
votes
1answer
152 views

autoscale static content on ec2 using local cache

Hello Serverfault members. I want to achieve the following setup on EC2: A load balancer starts and stops instances which only serve static content. (a javascript based application) I do not want to ...
0
votes
1answer
544 views

Amazon AWS Store Wordpress uploads on S3

I'm getting ready to move my company's website and order management application to AWS. We use wordpress as the 'face' of our company. Our order management app is for internal use. I'm using Elastic ...
0
votes
1answer
524 views

Upload database backup from mysql to Amazon S3 or Glacier without creating local file

Is there a tool that makes possible to backup a Mysql database to Amazon S3 or Amazon Glacier without having o create a local file with the database contents? Something like that: mysqldump -u ...
0
votes
1answer
252 views

configure s3fs mount for write-only

I am using s3fs to mount a S3 buckets on an Ubunutu server to store my daily backups there. Considering security this would allow a potential hacker to delete all my backups from the mounted bucket. ...
0
votes
1answer
504 views

How to find out what's missing from my iptables, why it's blocking my AWS transfer?

I use ManageWP to backup WordPress on my VPS then upload that to Amazon S3. It's failing with "Error 503 Service Unavailable" at the upload part when I have iptables running. When the iptables ...
0
votes
1answer
385 views

Amazon S3 - can't upload files into the same folder using s3sync.exe

I'm trying to upload a file using S3sync.exe into a certain folder in my S3 bucket. This is the command line : S3Sync.exe -AWSAccessKeyId accKey -AWSSecretAccessKey secKey -RequestEndpoint s3-ap-...
0
votes
1answer
253 views

s3 multiple write issue

I've built an application which forks process pools to fetch web content. I have 3 ec2 instances mounted onto one s3 bucket via fuse. I'm experiencing intermittent errors writing files. From what I ...
0
votes
1answer
176 views

S3 backup: bucket vs. volume

I've inherited a project that has been pushing encrypted (GPG) DB backups via a backup script to an S3 bucket. That's all been well and good, but now that bootable EBS instances (where the project ...
0
votes
1answer
571 views

S3sync not working

I managed to get s3sync to upload my test folder to Amazon S3 and can see it in the MWS Managment Console. Downloading the data back to a test folder results in the following error message: root@...
0
votes
1answer
69 views

Backing Up User Data when data is not in use. Should I be concerned?

This may be a dumb question. I would like to use duplicity to make backups to Amazon S3 of directories, each of which contains a different user's data. Each directory could be written to at any time. ...
0
votes
1answer
454 views

CDN Rerouting on 404 (file not yet in synch with original storage)

Here is the problem. I've setup my app(on EC2) to store uploaded images directly on Amazon S3. I'd like to be able to serve static files(cdn) from my 'home' server so I wrote script that does sync ...
-1
votes
1answer
18 views

Amazon S3: encryption speed and keys

I have couple theoretical questions about Amazon S3 I need to encrypt all my files. What encryption speed Amazon provide for S3? After encryption Amazon gives my key for file (or files package), it ...
-1
votes
1answer
462 views

AWS s3 - How to serve different file based on url parameter

For legacy reasons I need to serve different files from amazon s3 based on query string parameters. For example myfile.php?type=1 should serve other file than myfile.php?type=2. the files themselves ...