Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
1,134
questions
1
vote
1
answer
4k
views
URL Rewriting on a S3 Bucket
Is there anyway at all to setup URL rewriting for a static-hosted website on S3? I'm in the process of migrating over an existing website and some of the URLs I would like for them to have a 301 ...
1
vote
1
answer
13k
views
Nginx static content with Amazon S3
Im using Nginx front-end server in front of glassfish app. server. I uploaded my static contents to Amazon S3. I need to implement passing static request to Amazon's URL. I did a conf file like below.
...
1
vote
2
answers
96
views
Cloud storage options for client downloads
I'm looking for a way to distribute sample files to clients.
Requirement is that it's (relatively) simple to upload files, a link can be sent to a client who can download the file with a browser, the ...
1
vote
4
answers
400
views
Best practice for synchronizing sources across servers (Amazon EC2)
I'm moving from single app server to load balanced config.
In the single server config I synchronized the sources by S3 cmd sync s3 every few minutes and on hostup (Scalr event) I sync the sources ...
1
vote
3
answers
800
views
Is it secure to upload all kinds of file in website?
we have a website that allow the user to upload the files.Now if I am allowing the user to upload all the file extensions without restricting them is it secure?I am restricting only by the file size ...
1
vote
4
answers
1k
views
Good simple VPS to S3 Backup scripts?
anyone has a nice, simple script for backing up a VPS to a S3 bucket? can be ruby, python, whatever...
should be easy to configure!
thanks
1
vote
1
answer
659
views
Does librsync really reduce bandwidth for Amazon S3
I am looking to do some backups to S3 and want to reduce the bandwidth as much as possible. I am looking at a few options. It seems like librsync is the best solution for low bandwidth remote backups.
...
1
vote
1
answer
783
views
Where would an S3 upload speed cap originate?
I do a ton of uploading to S3 and am experiencing capped speeds and I can't quite figure out how to address it.
The setup: Windows Server 2008 R2 x64, external HD, using a Java based upload tool ...
1
vote
1
answer
2k
views
How to use Symantec Backup Exec with S3 Amazon Storage?
Is there any proven efficient way on how to use Symantec Backup Exect for Windows Servers to use S3 as one of the Devices to backup to. Even if it means to use a plug-in to make it work.
thanks!
1
vote
1
answer
51
views
How can I efficiently move an S3 bucket to a new AWS region while keeping the same bucket name?
My current plan to move an S3 bucket from AWS region us-west-1 to AWS region us-west-2 while keeping the same bucket name seems highly inefficient as it's copying the entire bucket twice:
# Create new ...
1
vote
1
answer
110
views
Is Mastodon remove-orphans safe when multiple instances sharing an S3 bucket?
I have two Mastodon v4.2.0 instances, a staging server and a live/production instance. They share an AWS S3 bucket for file storage. Both are hosted on DigitalOcean.
Is the command RAILS_ENV=...
1
vote
2
answers
2k
views
ansible - couldn't resolve module/action 'amazon.aws.s3_object' / The module amazon.aws.s3_object was not found in configured module paths
Per requirements, I did installed all the required packages, however amazon.aws.s3_object is one of those things:
couldn't be resolved/found
was not found in configured module paths
step(s), I used ...
1
vote
1
answer
533
views
AWS IAM: deny users from creating policies on specific resources
I want to be able to give my admin users the permission to create policies in IAM, but I want to make sure that they aren't able to create a policy that affects a specific resource.
To be more ...
1
vote
1
answer
1k
views
RDS PostgreSQL Import+Export to/from S3
Is it possible to enable both importing and exporting with an RDS PostgreSQL instance to an S3 bucket? I've been able to use the following pattern to enable one or the other with consistent success:
...
1
vote
1
answer
438
views
AWS s3 sftp without AWS transfer family
Are there any other ways to upload files from an Ubuntu server to AWS s3 without AWS Transfer Family? I have to make a MYSQL dump every five minutes and upload to the bucket but the Transfer Family's ...
1
vote
1
answer
940
views
Fargate task from service with Public IP disabled can't download env file from S3
We have a Fargate service that should be exposed to the internet via a load balancer, and since for tests we had used so far the random Public IP of the task, we decided to disable the Public IP, so ...
1
vote
1
answer
846
views
Universally apply restriction to bucket policy, for all S3 buckets
I want to apply a specific restriction to all S3 buckets. Generally AWS published resources describe a per bucket solution. However this does not solve the issue of other users creating bucket ...
1
vote
1
answer
266
views
Most efficient way to enable S3 versioning on existing objects?
The AWS S3 API lets you enable versioning on a bucket using the aws s3api put-bucket-versioning command. Future objects written to that bucket are versioned, but existing objects are not.
But is there ...
1
vote
1
answer
2k
views
What is the format of S3 lifecycle configuration?
My Problem
An S3 lifecycle configuration changes object storage classes. A typical usage is periodically moving objects to cheaper storage classes after a specified retention period.
I am trying to ...
1
vote
1
answer
286
views
Serving secure static S3 site without www
I'm trying to set up AWS to serve a static S3 site over SSL with a custom domain. I'm so close but I'm missing one piece.
S3 static website (works): xyz.s3-website.us-east-2.amazonaws.com
CloudFront ...
1
vote
1
answer
344
views
Setting up S3 to host 'wp-content' folder
If this is not the right SE to post the question, please kindly direct me to the right one.
I've seen configuration of setting up S3 to host wp-content folder, what are the advantages of doing so? ...
1
vote
1
answer
1k
views
Flask app on ECS that pull images from s3 bucket
I have a flask app that I want to pull images to display from an s3 bucket.
How do I implement the credentials parts? what I mean is:
Do I use boto3 with aws credentials to pull the images from the ...
1
vote
2
answers
2k
views
Digital Ocean Spaces S3: Is it possible to add access-control-allow-origin: * to all requests?
If you curl my image
curl -i https://local-spaces.fra1.digitaloceanspaces.com/test.jpg
you will see that there is no Access-Control-Allow-Origin header in the response.
If you curl like this, the ...
1
vote
1
answer
215
views
Can you limit the operations to log in s3 buckets?
I've been using the s3 logs to track downloads for a podcast (mp3 files) I currently have 23 files I actually care about and I only need to know information about GET operations (download requests ...
1
vote
1
answer
342
views
If I already have files in an S3 bucket in the Glacier storage class, what happens if I apply a bucket policy to move to Glacier after 1 day?
We have a pre-existing S3 bucket that we are looking to add a lifecycle policy to in order to move large files in S3 standard to Glacier. The bucket already contains items with the Glacier storage ...
1
vote
1
answer
254
views
move existing content in wordpress to s3
I have a existing wordpress website in Elasticbeanstalk AWS with auto scaling and load balancing and external RDS. while scaling policies are applied I will loose my existing data in EC2 instances so ...
1
vote
1
answer
1k
views
Clever way to reduce rsync/rclone load with backups to Amazon S3?
I started running rclone to keep a set of files on my laptop that are synced up to the cloud and encrypted. I wanted to 'control' the encryption because of trust issues. Rclone does this pretty well, ...
1
vote
1
answer
2k
views
Editable URL Rewrite Map for CloudFront/S3 Website using Lambda@Edge
I'm migrating a static website from IIS to Amazon S3, using CloudFront for https. The old site used the URL Rewrite module and had 301 redirect maps with over 200 entries. Here is an example:
example....
1
vote
1
answer
1k
views
Files Not Appearing in S3 File Storage Gateway Bucket
I created an AWS S3 File Storage Gateway and mounted it to a Linux server on-prem. On that same server where the file gateway is mounted I have a separate NFS share mounted. My goal is to copy files ...
1
vote
1
answer
122
views
Restore AWS Storage Gateway Server
I just setup an AWS File Storage Gateway server on-prem. I was able to connect it to an S3 bucket and I can mount it as an NFS share to Linux servers which is great. My concern though is this - if for ...
1
vote
1
answer
618
views
S3 Bucket for hosting media, s3fs mount catch-22
I'm attempting to use an S3 bucket to host media files (images, video and static files) for a small web application and came across a strange catch-22...
According to the S3 documentation, to serve a ...
1
vote
2
answers
64
views
Https configuration on AWS [closed]
I have an angular application hosted on aws s3, say example.com. The url is hosted on route53. You can access my website in non https here: http://proscorepad.com/
I created two repository that are ...
1
vote
1
answer
5k
views
cannot get aws cloudfront / route53 to resolve my domain name
I have setup a cloudfront distribution with SSL, pointing to a s3 bucket which is public and setup as a static website. I chose the option to redirect HTTP to HTTPS, as I want to force HTTPS.
I ...
1
vote
1
answer
290
views
Multiple server backups using kubernetes and rsnapshot
I am looking to setup multiple backups, from multiple servers in different geographical locations at different times using something like a docker/rsnapshot and kubernetes (master + a couple nodes at ...
1
vote
1
answer
1k
views
Minimum AWS policy actions required for retrieving Glacier files through S3?
I'm trying to create a policy for an IAM account that will allow an employee to have full read permissions for our S3 (lifecycled to Glacier) buckets, with no unnecessary write abilities as to avoid ...
1
vote
1
answer
738
views
AWS, S3 + VPC Endpoints
Not sure what I'm missing here.
I've got an S3 bucket. I've got an EC2 instance.
I created a VPC endpoint and associated it with ALL the subnets in my VPC, with the default policy:
{
"Statement": ...
1
vote
1
answer
2k
views
What do cloudwatch s3 metrics count as an object?
I was recently using the metrics under the management tab in the aws s3 console and notices that for a few of my buckets the NumberOfObjects (count / day) graph seemed to be way off. I had roughly 50 ...
1
vote
1
answer
2k
views
Subdomain from different S3 bucket on same cloudfront distribution
I have example.com served from idxxxx.cloudfront.net, this goes to S3 bucket nr 1. This works well.
I now need subdomain.example.com to also be served from same cloudfront distribution, and point to ...
1
vote
1
answer
826
views
AWS CloudFormation: Apply S3 Bucket delete Policy to all buckets
I am trying to find a way to provide an additional level of protection to keep S3 Buckets from accidentally deleted. I want to implement bucket policies, but I want to do it through cloudformation so ...
1
vote
1
answer
301
views
Goodsync S3 Empty Secret Access Key (AKA Hacking you Linux Goodsync configurations file)
I'm trying to use Goodsync from Linux (Wheezy). I have set up a backup to S3 (see below) but I keep getting the error Empty Secret Access Key.
=== Job === test:<
'file:///testgsynclocal/' <-&...
1
vote
2
answers
4k
views
Cloud-init is failing to execute awscli command in User Data
Need help to fix the error and enable the instance to run issue free User Data script.
Issue: cloud-init is unable to execute awscli command. I have mentioned a custom script (which will be present ...
1
vote
1
answer
2k
views
Set up SES to receive to S3 bucket, not getting anything
NB: I'm an AWS noob.
I followed the instructions to set up SES receiving (no sending needed) for my domain to write incoming emails to an S3 bucket. The domain is managed through Route 53 & ...
1
vote
1
answer
296
views
CTRL+C on 200k files aws s3 sync
I'm syncing a directory containing 200k files (140GB) to a bucket using the following command
aws s3 sync target s3:/awesomebucket/key/prefix/ --region ap-southeast-2
I took along time. The network ...
1
vote
2
answers
130
views
Optimized S3 Policy for full access
I have the following policy in place:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowUserToSeeBucketListInTheConsole",
"Action": [
"...
1
vote
1
answer
937
views
Moodledata on shared storage on AWS
I am in the process of setting up a simple Moodle cluster on AWS for production use. The plan is to use a loadblancer, two EC2 instances and a RDS. I have worked extensively with Moodle before but ...
1
vote
1
answer
750
views
Best manner to access files in S3 Bucket from EC2 instance
I have tried several packages (goofys, s3fs, fuse) and have not been able to find a solution to my problem.
I have an S3 Bucket with several thousand files currently in it and will continue to grow. ...
1
vote
1
answer
1k
views
Is not authorized to perform an operation while I have the full access of an aws service
I have created a simple nodejs script to handle the files in S3, and I want it to be executed on lambda. My admin has added me in IAM with full access to lambda and S3. However, when I select the "S3 ...
1
vote
1
answer
86
views
What is the right way to back up google authenticator emergency keys?
I'm implementing Google 2-Factor Authentication on some servers in my company.
When configuring Google 2FA on client computers, 5 emergency codes are generated to be used if a user forgets his master ...
1
vote
1
answer
708
views
duplicity s3 backup from centos
I'm trying to backup server data to S3 but getting no success. Below is the script which I run to start backup on S3.
I have create a user using IAM and have granted all permissions to the user.
#!...
1
vote
1
answer
919
views
EC2 to S3 upload speed decrease with increase in instances
I have a job that gets data from a database, runs some code, and uploads the result to an S3 bucket. The code takes approximately 1 minute to run, and the result file is approximately 10MB. Both the ...