This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

0
votes
2answers
20 views

How to configure AWS Route 53 to get equivalent of domain forwarding with masking

(I apologize in advance for not having enough technical depth to ask this intelligently) I currently have an AngularJS static website hosted on AWS S3. I'm using AWS Route 53 for DNS. I'm able to ...
3
votes
3answers
1k views

projects or packages to emulate a tape drive with a Amazon S3 or OpenStack Swift compatible backend?

I've got some data that is being backed up to tape via bacula, and at some point soon I am going to need to migrate to non-tape storage. Bacula supports to-disk archiving, but it occurred to me that ...
0
votes
2answers
1k views

How can I set up an SFTP server backed by S3 (or similar)

I need to set up an SFTP server that, essentially, has very large capacity. I need to give one of our partners SFTP login details to a server where they will upload millions of files, totalling a few ...
2
votes
1answer
20 views

AWS: Why is the bucket size on CloudWatch higher than when using S3 tools?

I use the awscli tools to verify the size of my buckets. First the S3 call (all files without versioning, just an ordinary upload): $ aws s3 ls s3://XXX --recursive --human-readable --summarize ...
0
votes
0answers
11 views

looking for index.php while proxy_pass to s3 bucket

I am configuring my nginx proxy server to fetch data from s3 bucket in case of 502 error. This is my configuration file. error_page 502 @static; location @static{ rewrite ^/a/b /a_b break; ...
0
votes
0answers
27 views

Access denied for the request of data from s3 bucket on high load

I am trying to fetch static files from s3 bucket in case of 502 error. This is how I have configured nginx for requests : error_page 502 = @static; location / { index index.php; try_files $uri ...
2
votes
2answers
609 views

Getting files from an s3 bucket using IAM role credentials

I am trying to retrieve some files from a private s3 bucket to a filesystem location elastic beanstalk ec2 instance, but with no success. I've created a bucket named dev-config containing a file ...
0
votes
1answer
15 views

AWS backup on another attached drive

In AWS-EC2, I was thinking of backup my code data and database in RDS via bash script on separate attached drive on same instance. I want to do it like that way so it becomes easier for me during ...
2
votes
1answer
945 views

Best failover strategy for e-mail servers on AWS to ensure high availability

We have our e-mail server hosted on AWS. Last week Amazon had a failure in their East Coast region which brought down our server along with many others. We now want to implement a failover strategy ...
5
votes
1answer
136 views

How can I tell where an Amazon AWS key is being used?

I have inherited an Amazon AWS environment in which the Root account key has been widely distributed for the purposes of making backups to S3 buckets. I need to track down where the key is being ...
0
votes
1answer
29 views

Elastic Beanstalk S3 access via .ebextensions

I have a simple file in my .ebextensions folder: 00-myconfig.config Resources: AWSEBAutoScalingGroup: Metadata: AWS::CloudFormation::Authentication: S3Access: ...
-1
votes
0answers
4 views

Use s3-parallel-put on Windows

I need to move a large number of files from a Windows EC2 instance to S3. How can I use s3-parallel-put or something similar to do this? I have attempted to use s3-parallel-put already with errors.
0
votes
0answers
8 views

CloudFront + S3 Access Denied Using Signed Cookies

I'm trying to integrate signed cookies for a CloudFront distribution that's fed off of an S3 bucket but getting access denied error messages: <Error> <Code>AccessDenied</Code> ...
0
votes
1answer
27 views

when to use NFS for web app

I am currently building a simple webapp architecture on AWS, as follow: Elastic Load Balancer - multiple front ends - RDS database. I have also set up an NFS server to store all images uploaded by ...
0
votes
0answers
9 views

Amazon S3 - Upfront File Organization Architectural Decisions

I'm currently building a web application for a publishing platform, and I am using S3 for content storage. S3 is super simple to use, but I'd like to try to make the best possible file organization ...
-1
votes
0answers
22 views

Amazon Route 53 -> S3 Mask Root Alias URL [duplicate]

I'm new to DNS configurations, so please bear with me if my terminology is a bit off. I am using Amazon Route 53 for DNS to point my domain name to a static website hosted on Amazon S3. I have an A ...
0
votes
2answers
25 views

How to transfer data between s3 buckets of the different aws account using command line

How to transfer data between s3 buckets of the different aws account using s3cmd ? command will be like this: s3cmd rsync s3://acc1_bucket/folder/ s3://acc2_bucket/folder --recursive But then how ...
2
votes
1answer
1k views

Amazon Cloudfront with S3 Redirect

I'm trying to do something that should be relatively straight-forward - I want to setup a few domains and subdomains to redirect to the core domain for our site, but I want to put the redirects out on ...
1
vote
2answers
1k views

s3fs Input/output error

I'm trying to set up a backup system with s3fs and the Amazon S3 service. I followed this this guide and this guide. Tailing /var/log/messages I get: Aug 28 13:37:46 server s3fs:###response=403 I ...
1
vote
2answers
831 views

regex to match s3 urls in nginx location directive & proxy to amazon s3

In nginx location directive, how do I match s3 urls. for example, the wrong url is: http://example.com/https://s3.amazonaws.com/mybucket/logo.jpg?1404251306 from Logs, I can see nginx can that it ...
0
votes
2answers
452 views

No permissions for files in Amazon s3 for admin

We transferred files from one bucket to another bucket on a separate account with --acl-public as follows: s3cmd cp -v --recursive --acl-public s3://source-geo/webroot s3://dest-geo/webroot/ Now we ...
1
vote
1answer
1k views

Using TntDrive to mount Amazon S3 on a Windows Amazon EC2 instance

I currently have 2 Amazon EC2 Windows Server 2008 instances setup with a load balancer. My problem now is that I need both instances to be able to access a single file system source (I'm trying to use ...
1
vote
1answer
34 views

S3 Static Website hosting - restricting access from IP Range

Is there way by which can we enforce the access only from a Particular IP Range for S3 Static Website hosted site ?
0
votes
2answers
974 views

Enable Cross Origin Scripting on Nginx from retina.js to load files from another URL?

Im running a retina image plugin along with W3 Cache for CDN support. W3 cache moves images to S3, but the JS file won't server the files from there due to the difference in URL (it goes from h ...
3
votes
1answer
3k views

s3put fails with “Connection reset by peer” for large files

I can upload small files quite fine using the same commands, however whenever I try to upload a 5 GB file to S3 then the s3 script fails: [backup]$ s3put --bucket john-murdoch-bucket --reduced ...
0
votes
0answers
26 views

logstash cloudfront codec plugin: Error: Object: #Version: 1.0 is not a legal argument to this wrapper, cause it doesn't respond to “read”

Logstash version 1.5.0.1 I am trying to use the logstash s3 input plugin to download cloudfront logs and the cloudfront codec plugin to filter the stream. I installed the cloudfront codec with ...
1
vote
1answer
26 views

Logrotate script not uploading to s3

I am trying to use logrotate to upload log files from my rails and nginx commands once a day. When I run the logrotate command manually the files successfully upload, but it never does it on its own. ...
0
votes
0answers
11 views

How to use the s3cmd setlifecycle command?

s3cmd has a setlifecycle subcommand that seems like it ought to let me configure a bucket into Amazon Glacier. However I can't find any documentation for it on the web or in the manpage. What's the ...
-1
votes
1answer
23 views

How can I easily backup multiple folders on EC2 to S3

I move dockers around a lot on EC2 instances. Each docker has a specific folder on the host where it stores it's data. I want to create a single backup script that will look for any of a list of ...
80
votes
19answers
58k views

How can I get the size of an Amazon S3 bucket?

I'd like to graph the size (in bytes, and # of items) of an Amazon S3 bucket and am looking for an efficient way to get the data. The s3cmd tools provide a way to get the total file size using s3cmd ...
1
vote
1answer
82 views

AWS S3 bucket permissions

I've created AWS s3 buckets with several thousand objects, initially the access policy was restricted to authenticated users, but later due to some requirement we need to make all objects in this ...
1
vote
1answer
122 views

jboss cluster in ec2 with s3ping

Am trying to get s3ping discovery method working in wildfly 8.0.0. I am using default Amazon Linux AMI and using chef to launch instances in ec2 based on this tutorial ...
0
votes
0answers
17 views

aws hosted website not serving web pages to internal network and serving from s3 bucket instead

Sorry about more text to describe..(The issue has been resolved but the causes are un-known) We Have one website hosted via AWS Load Balancer and was working from last year perfectly. Recently an ...
2
votes
1answer
491 views

How can I whitelist oubound-from-private-subnet traffic to S3 on the NAT instance of an EC2 VPC?

I'm researching privacy-compliant high-security deployment options for AWS VPC. I'm looking for a method of tightening outbound traffic from the private subnets at the NAT instance. For example, how ...
2
votes
1answer
78 views

Why would a Image Hosting website, such as Imgur, use AWS EC2 instances over S3 buckets for uploads?

I was reading a Q&A with the creator of Imgur, and he went into detail explaining the server infrastructure that Imgur runs on. Here's a small quote from what he had to say: Most of the ...
0
votes
2answers
50 views

S3 backup with versioning plus lifecycle management to prevent malicious deletion of backups?

I just want to run this idea by some smarter people to make sure I'm not overlooking something obvious: I want to backup my Linux server to S3 using one of the many backup scripts that allow ...
1
vote
1answer
66 views

Locating an AWS S3 physically in a private subnet (like RDS)

Similar to how an RDS instance requires you to pick the specific subnets (which can be in a VPC's private subnet), can AWS S3 stores also be physically located in a VPC's private subnet?
5
votes
1answer
6k views

How to force s3fs mount on boot

I'm using s3fs 1.33 on Ubuntu 9.10 x86. I mount it manually with this command: sudo /usr/bin/s3fs mybucket /mnt/s3/mybucket -ouse_cache=/tmp How can I force the mount on boot? I guess that I have ...
2
votes
1answer
201 views

s3 policy for quota

Is there a s3 policy rule that can only allow a user to upload so many GB of files? Basically looking to set a quota on the objects a user can upload. Here is the rule set that I have now: { ...
3
votes
3answers
678 views

Amazon AWS EC2 instance Exported to local machine

I have several Amazon AWS EC2 virtual machines that I created on AWS EC2. They were not imported, they were created there. I would like to export these machines in order to achieve local backups, ...
13
votes
6answers
15k views

How can one efficiently use S3 to back up files incrementally?

I understand how rsync works on a high-level, but there are 2 sides. With S3 there is no daemon to speak of — well there is, but it's basically just HTTP. There look to be a few approaches. s3rsync ...
1
vote
1answer
29 views

Why is AWS S3 Redirect Showing as 302 instead of 301?

I'm implementing a 301 redirect in S3 using this blog post from when the functionality was first announced. It explicitly says the redirect will be a 301, which is what I want. Screenshot of the ...
0
votes
1answer
632 views

s3fs “could not determine how to establish security credentials”

I'm trying to use s3fs to mount an S3 bucket on to a standard AWS Amazon Linux AMI (with all the necessary dependencies installed). However when following this tutorial when I run: s3fs mybucketname ...
13
votes
4answers
10k views

Which is the fastest way to copy 400G of files from an ec2 elastic block store volume to s3?

I have to copy 400G of files from an elastic block store volume to an s3 bucket... Those are about 300k files of ~1Mb I've tried s3cmd and s3fuse, both of them are really, really slow.. s3cmd ran for ...
3
votes
0answers
76 views

Set up a different index document per directory with S3 static website hosting

I am using S3 static website hosting, and trying to get the following redirects to work: example.com --> example.com/index.html example.com/foo --> example.com/foo/foo.html example.com/bar ...
-2
votes
1answer
27 views

Sync files between AWS S3 and Rackspace cloudfiles [closed]

I am looking for a solution for syncing files between AWS S3 and Rackspace Cloudfiles on a regular basis. Preferably, using a third party service or low maintenance self-hosted solution.
0
votes
0answers
20 views

How to securely allow web apps to read/write to S3? (the new way with IAM)

I've been banging my head against a wall on this for two hours. It's frustrating that Amazon has made this so complicated when it doesn't need to be. Let's say I have a bucket called "test," and a ...
0
votes
1answer
53 views

Using Duplicity with AWS S3 and IAM Folder policy

I'm trying to setup backups from multiple hosts to a single bucket. Each host has their own AWS login and credentials. The top most folder (prefix) maps to the AWS user id and I'm trying to limit ...
6
votes
6answers
15k views

How to schedule automatic (daily) snapshots of AWS EC2 Windows Instance?

I have some Windows servers hosted on Amazon EC2. Some run Windows Server 2003 and other run Windows Server 2008. These are EBS-backed instances. Most of the instances also have some additional ...
1
vote
2answers
1k views

Anonymous access to S3 bucket only from my EC2 instances

Is there a way to let anonymous access to a certain S3 bucket only from my EC2 instances (all of them) within a single AWS account? I know I can use IAM roles, but I found it's just too many moving ...