This is for questions on Amazon's cloud storage service.

learn more… | top users | synonyms (1)

0
votes
0answers
10 views

How to securely allow web apps to read/write to S3? (the new way with IAM)

I've been banging my head against a wall on this for two hours. It's frustrating that Amazon has made this so complicated when it doesn't need to be. Let's say I have a bucket called "test," and a ...
-1
votes
0answers
5 views

Set up a different index document per directory with S3 static website hosting

I am using S3 static website hosting, and trying to get the following redirects to work: example.com --> example.com/index.html example.com/foo --> example.com/foo/foo.html example.com/bar --> ...
0
votes
0answers
30 views

DNS Based Load balancing on squid - odd issues

I'm attempting to do round-robin DNS based load balancing for two squid boxes running as simple HTTP(S) proxies. For testing I'm using AWS Route 53. To simulate round-robin I created two records ...
0
votes
0answers
10 views

Duplicity for remote backup to AWS S3

I am using Bacula with S3 for backups. I keep bacula volumes for 60 days or so. Using duplicity as I would like to use GPG encryption of the volumes. Is there any way to do an rsync style sync using ...
-2
votes
1answer
42 views

Use Apt Inside AWS VPC Without NAT Instance

I have several Ubuntu machines inside VPCs. For many of them, the only network access they need whatsoever is for "apt-get update && apt-get upgrade" and NTP. For NTP, I can simply have an NTP ...
0
votes
1answer
33 views

How best way to rsync from aws s3 to gcloud?

I must execute a rsync ( not only copy or move ) from all my buckets from s3 to google cloud. I need create the same infraestructure of s3 to google. I have like 2Tera in s3, and all days I receive ...
0
votes
0answers
31 views

AWS security group egress rules for S3

I manually created a new security group using the AWS CLI. I created ingress rules that allow incoming connections only from my company's public IP address using the known ports for SSH (22) and ...
0
votes
1answer
45 views

Is S3 a proper choice to keep live linux user home directories?

We are building an RDP cluster using Amazon EC2/S3. Users' home directories are on S3 (single bucket) and all EC2 nodes mount the same S3 bucket using S3FS. Each EC2 node uses the appropriate home ...
0
votes
1answer
22 views

Can I use an IAM role to grant my Heroku app access to my Amazon S3 bucket?

Heroku: Using AWS S3 to Store Static Assets and File Uploads suggests using my AWS security credentials to enable my Heroku app to access my Amazon S3 bucket. However, isn't it better practice (as ...
0
votes
0answers
16 views

Duplicity GPG key not available

I am using the following tutorial to get Duplicity to back up my mysql dumps to Amazon S3. http://kappataumu.com/articles/cloud-backups-duplicity-s3.html However I am getting an issue with the ...
0
votes
1answer
25 views

AWS/Route53/S3: Redirecting www.example.com to example.com works. Redirecting example.com to www.example.com doesn't

To set up a static site on S3 with DNS on Route53, we have two options. If we want the users to type www.example.com and have that auto-replaced in the address bar by example.com, we would do the ...
0
votes
0answers
15 views

Uploaded 0 object(s) to bucket from Write-S3Object command

When I upload a file with the Write-S3Object command and it seems work properly and in another tool I can see the file but I get the follow message: PS> Write-S3Object -BucketName my-bucket ...
0
votes
1answer
48 views

Amazon Cloud Formation: Import file from S3 bucket

I am struggling to get files from my S3 bucket to my EC2 instance using cloud formation. I have the following files: "files" : { "/var/www/text.txt": { "source": ...
0
votes
0answers
12 views

Using Duplicity with AWS S3 and IAM Folder policy

I'm trying to setup backups from multiple hosts to a single bucket. Each host has their own AWS login and credentials. The top most folder (prefix) maps to the AWS user id and I'm trying to limit ...
1
vote
0answers
12 views

S3 + Cloudflare vs S3 + 2 Web Server + Cloudflate

Currently, I run two websites and each site own it own big server. Currently, all of those two sites has roughly 300GB of images. Its grow rate is 15GB of images per month. I know that eventually, one ...
0
votes
0answers
29 views

EC2 websites & RDS databases backup

We have some Amazon EC2 machines on which different websites are working, and their database is being pulled from RDS-MySQL, I know for machine I can have images as backup and for RDP-MySQL it has ...
0
votes
0answers
99 views

Nginx as s3 proxy with private buckets

I'm trying to set up Nginx as a proxy for a static html/js webapp hosted on S3. I read a lots of tutorials and posts and I could make it work if my bucket is set as public. The reason I need Nginx as ...
0
votes
1answer
26 views

How to stop s3 from listing some of the content?

It is an example. What setting of S3 causes the s3 server to a listing of bucket content? EDIT: I have followed Céline Aussourd's answer and turned off the list of the s3 bucket. But ...
0
votes
0answers
59 views

duplicity s3 backup from centos

I'm trying to backup server data to S3 but getting no success. Below is the script which I run to start backup on S3. I have create a user using IAM and have granted all permissions to the user. ...
0
votes
1answer
43 views

Locating an AWS S3 physically in a private subnet (like RDS)

Similar to how an RDS instance requires you to pick the specific subnets (which can be in a VPC's private subnet), can AWS S3 stores also be physically located in a VPC's private subnet?
0
votes
1answer
16 views

What is the easiest way to duplicate an existing s3 bucket (excluding data)? [closed]

I want to be able to create a new bucket which mirror the settings of an existing bucket. What is the best way to do it? Ideally I want to be able to capture the settings via aws cli and create a ...
0
votes
0answers
12 views

How can I improve the space usage of duplicate content in S3?

In my project I have duplicated files in various folders in a S3 bucket. Based on the output of s3cmd du, each time I make a copy of some content in S3, the usage increases. I wonder does s3 support ...
0
votes
0answers
8 views

Read data from external bucket on Amazon EMR

I need to read data from a bucket that I don't own, and I get the IAM account key and secrete access key to the bucket. What is the best way I can access the data? How can I add another IAM with ...
0
votes
1answer
67 views

EC2 to S3 upload speed decrease with increase in instances

I have a job that gets data from a database, runs some code, and uploads the result to an S3 bucket. The code takes approximately 1 minute to run, and the result file is approximately 10MB. Both the ...
1
vote
0answers
56 views

AWS S3 bucket permissions

I've created AWS s3 buckets with several thousand objects, initially the access policy was restricted to authenticated users, but later due to some requirement we need to make all objects in this ...
3
votes
2answers
97 views

Replicated shared filesystem

I'm looking into setting up a shared filesystem/file server on AWS (EC2) infrastructure that offers replication and fairly painless failover. This filesystem would host potentially millions of files ...
1
vote
0answers
56 views

jboss cluster in ec2 with s3ping

Am trying to get s3ping discovery method working in wildfly 8.0.0. I am using default Amazon Linux AMI and using chef to launch instances in ec2 based on this tutorial ...
0
votes
1answer
30 views

Origin access identity and S3 permission

From http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html , I have already created an OAI, as shown in this screenshot Am I supposed to grant the permission to ...
0
votes
1answer
117 views

Copy directory structure intact to AWS S3 bucket

I want to use the AWS S3 cli to copy a full directory structure to an S3 bucket. So far, everything I've tried copies the files to the bucket, but the directory structure is collapsed. (to say it ...
1
vote
2answers
64 views

Most efficient way to batch delete S3 Files

I'd like to be able to batch delete thousands or tens of thousands of files at a time on S3. Each file would be anywhere from 1MB to 50MB. Naturally, I don't want the user (or my server) to be waiting ...
0
votes
1answer
34 views

s3 service using s3cmd tool through URL based firewalls

I am behind a URL based application firewall and currently using s3cmd tool to access amazon's s3 service. Doing a bit of searching has asked me to whitelist s3.amazonaws.com but I see that s3cmd uses ...
0
votes
1answer
25 views

AWS S3 Sync options duplicates / weird behaviour

I am trying to sync a folder with 2M+ files to S3, everything went OK but then 40,000 files were not uploaded (server crashed randomly), when I tried to do a sync command again, it started from 0, ...
0
votes
1answer
82 views

Where are AMIs stored?

It seems like an very simple question, but yet I could not find the answer. When I make an instance on Amazon AWS and then make some changes and create an AMI where is that AMI stored? S3? Will making ...
0
votes
2answers
137 views

How to restrict outbound EC2 to only access S3?

I want to restrict the outbound security group from an EC2 instance. The instance only needs to access an S3 bucket. I just learned that S3 uses port HTTPS (443). I could just put that rule in place ...
2
votes
0answers
36 views

Most bandwidth efficient setup for video sharing [closed]

I built a video sharing platform for a popular french podcast I contribute to. Initially I used S3 and linked directly to it, bad idea. Then I used a pull CDN to better optimize my bandwidth, but s3 ...
0
votes
1answer
507 views

How can I set up an SFTP server backed by S3 (or similar)

I need to set up an SFTP server that, essentially, has very large capacity. I need to give one of our partners SFTP login details to a server where they will upload millions of files, totalling a few ...
0
votes
1answer
43 views

Limit my S3 bucket read access to EC2 instances run by others?

For instance, if I store data in US Standard, the S3 Pricing Page indicates that it is $0.00 per GB transferred out to Amazon EC2 in the Northern Virginia Region. And it looks like GET requests are ...
3
votes
1answer
229 views

CloudFront S3 Access denied

I have one account with privilege to manage S3 (S3_User), and another one to manage Cloud Front (Cloud_Front_User). Now, I've created a bucket "bucketname" with S3_User and a distribution with ...
-1
votes
1answer
32 views

Route 53 subdomains and their redirects

I am having problems with my redirects. I have a bucket named example.com created in s3 which redirects to http://google.com (just an example). In route 53 I am able to point my example.com domain to ...
2
votes
2answers
266 views

Getting files from an s3 bucket using IAM role credentials

I am trying to retrieve some files from a private s3 bucket to a filesystem location elastic beanstalk ec2 instance, but with no success. I've created a bucket named dev-config containing a file ...
0
votes
1answer
87 views

Direct multiple domains to specific URLs under one domain: Amazon S3 or Apache

I have a site with user profiles and following address structure: www.mywebsite.com/user1 www.mywebsite.com/user2 I need to allow users to point/redirect their own domains to their respective page ...
0
votes
1answer
30 views

Amazon S3 Versions: log one line per version

Before I go down this rabbit hole, has anybody tried this? Is it crazy? Let's say I want to put a log file in Amazon S3. But S3 doesn't have the ability to append to an existing object without first ...
2
votes
1answer
595 views

Access denied copying files using S3 CLI

Attempting to pull down the contents of an S3 bucket using the AWS CLI, I'm getting the following: aws s3 cp --region us-east-1 s3://s3.amazonaws.com/my-bucket . --recursive A client error ...
0
votes
1answer
53 views

Transfer enterprise data - tiny volumes - to AWS S3 for Heroku app

What would you suggest is the best option to get "enterprise" data from a client into AWS S3 so we can provide Heroku hosted SaaS? The data volumes are tiny, it's just daily spreadsheet/csv records. ...
3
votes
1answer
104 views

Deny anonymous downloads in AWS S3

I have a bucket with several files publicly downloadable. I want to put up a bucket policy after which these files should only be downloadable by my IAM users. The policy I have got so far is this: { ...
2
votes
0answers
68 views

Solution for file data backup, versioning and replication

We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
1
vote
1answer
153 views

How do I use SSL with Amazon S3 CNAME masked buckets?

I am successfully running an Amazon S3 bucket as media.mydomain.com. The problem is, I'm running an https site and I'm unable to link to files using https//media.mydomain.com. How can I switch to ...
3
votes
2answers
158 views

Move static files from EC2 to S3 (or cloudfront) without updating links

Right now, I have a site entirely on an EC2 instance including all static files like css, js, and images. I'm wondering about the best way to handle moving them. For example, in my <HEAD> on a ...
0
votes
1answer
90 views

Sync files from local filesystem with S3 - one way

I have a process that generates log files to the local filesystem, and files that are more than 2 weeks old are automatically purged. I want to use the aws command line tools to sync my folder with ...
2
votes
1answer
40 views

Does the AWS CLI mv command also do a copy-and-delete?

I am investigating renaming bundles in AWS S3. I would prefer to use the SDK, but it appears that the SDK only supports Copy and Delete. This because the object keys are used for partitoning the ...