Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
990
questions
1
vote
1answer
33 views
Amazon SES data via AWS S3: Is there a simple way to list and download a folder and get line counts?
I've set up Amazon SES to send a company announcement to a list of about 1,000 contacts. I set up the Kenesis Firehose to log all e-mail events (e.g., Send, Bounce, Click) to a bucket in S3. SES seems ...
0
votes
1answer
17 views
Trying to revoke public access on CloudFront S3 bucket
All the AWS security tools are bugging me all day about having public S3 bucket access turned on. So I said sure, the only bucket we have that needs public access is our cloudfront bucket, let me fix ...
-2
votes
1answer
26 views
Domain Double Pointing
I have one main domain basicdomain.com;
About 100 domains's CNAME are pointing to that main domain.
domain1.com, domain2.com, ... domain100.com
All domains will show basicdomain.com contents.
I am ...
0
votes
1answer
19 views
What kind of service I need to point multiple domains to different webpages?
I would like to provide my users a way to publish their site, meaning I'll be hosting it and I would provide them the DNS information so they can point their custom domain to it.
I'm totally new on ...
0
votes
0answers
24 views
S3 “folder-per-client” permissions strategy
I was originally planning on an S3 bucket per client, with bucket policies restricting access to that bucket to a single IP (the clients' single-tenant server). However, most S3 compatible service ...
0
votes
0answers
41 views
Confluent Kafka Connect S3 re-processing data due to offset out of range
I'm experiencing a strange situation.
My Confluent Kafka Connect S3 is processing around 8K messages/sec. What started happening randomly every few days is:
The Fetcher class would try to fetch an ...
0
votes
0answers
15 views
AWS Route 53 configuration problems for S3 hosting
I have a static site hosted on S3, mylifeinlimbo.ca
I have set up S3 with a bucket for www.mylifeinlimbo.ca which redirects to the main bucket and created an 'A' record in Route 53 (see image for ...
0
votes
1answer
15 views
CloudFront distribution from S3 bucket: big “bucket is public” warning. What can go wrong?
I'm moving an existing static website to S3/CloudFront.
Following some documentation from AWS, I setup a policy on the S3 bucket to make it public:
{
"Version": "2012-10-17",
"Statement": [
...
0
votes
0answers
53 views
Is there a way to make CrossFTP or DragonDisk access path-style s3 server?
I'm trying to find a way to speed up the transfer of 95gb of data to my company's Object Storage.
I'm transfering the files via node/aws-sdk but it's too slow.
I already tried ManagedUpload and ...
0
votes
1answer
25 views
Permanent redirect all http requests to include www subdomain [duplicate]
I have noticed both google.com and amazon.com, redirect requests to www.google.com and www.amazon.com
I am thinking of doing the same for my website which is hosted on AWS. What is the best approach ...
0
votes
1answer
20 views
AWS S3 and Bandwidth billing by Cognito user
I Want to build an serverless application that can store simple objects in S3, manage the user Login using AWS Cognito, and make the pricing pay-as-you-go for the user, (Charge the user for the ...
0
votes
0answers
19 views
Static websites in s3 bucket returns 404 for links
I'm uploading Gatsby static built files to s3 static website hosting in subdir.
For example.
test.s3-website-us-west-2.amazonaws.com/site1
test.s3-website-us-west-2.amazonaws.com/site2
So I want ...
0
votes
1answer
17 views
Cloudtrail S3 data events: “The maximum number of S3 data resources is 250”
We need to monitor object-level access and changes in one of our S3 buckets with cloudtrail (specifically with cloudtrail, not s3 access logs).
The cloudtrail documentation indicates in a small note ...
0
votes
0answers
25 views
Access denied for S3 bucket with redirect access via CloudFront
I have public S3 bucket exposed via CloudFront to S3 A record for www.mezinamiridici.cz.
https://www.mezinamiridici.cz/ - works
http://www.mezinamiridici.cz/ - redirects to https://www.mezinamiridici....
0
votes
0answers
33 views
Why does my S3 static website use the wrong content-type for the index document?
I have setup an S3 bucket with static web hosting enabled. The index document is set to index.html. index.html has the ContentType of text/html.
When I use bucket.s3-website-ap-southeast-2.amazonaws....
0
votes
0answers
19 views
My React Code is giving a strange response
As beginner with React.
I am making the test app for showing fullsize image when user click thumbnail image.
My App's Thumbnail Page
But when I click thumbnail image, strange S3 path returned. So I ...
0
votes
0answers
23 views
S3 ignoring Content-Disposition metadata when Content-Type header is set to “text/html”
We have a file in S3 that is a HTML file. We are trying to make it so that the HTML file downloads rather than renders by adding metadata to the object in S3 Content-Disposition: attachment. The ...
0
votes
0answers
20 views
Did AWS stop allowing PUTs to S3 locations more than one directory deep through the AWS CLI?
On our various servers that have a product that includes a database, I create a job that runs a small script that makes a backup of the database, compresses the file, and sends it to S3. I have one ...
1
vote
1answer
85 views
AWS S3 Bucket Django 3.0 User Profile Image Upload Access ERROR
INTRO
I am following this guide as recommended, here is the guide's GitHub repo.
I have created an AmazonS3FullAccess to it as well
I am using the guide's 3rd example "Mixing public assets and ...
1
vote
2answers
109 views
mysqldump using a lot of space
I have a server and the / partition is 20GB in size.
Databases are stored in /mnt/mysql-data partition is 500GB in size.
Now here's the problem. Whenever I run mysqldump it fills up / partition to ...
2
votes
2answers
35 views
Trigger AWS Lambda by S3 object GET
My Problem
I have a public S3 bucket which serves as a storage solution for large files (GBs) that I send to colleagues and clients. The files are auto-deleted after a few weeks.
The problem with ...
0
votes
0answers
13 views
Amazon S3: How to automatically copy uploaded files to another directory upon file creation
I have an S3 bucket where files are uploaded regularly, with the folowing directory structure:
backup/
process/
The ideia is: I upload files to backup, the new files are regularly copied to process ...
0
votes
0answers
16 views
How to make new uploads non private by default, i.e., respect the “public” policy on the bucket?
I have a new bucket where I am uploading, each day, new audio files to be consumed by my blog. I only want the files to be available when using them from my website, so I have a corresponding policy ...
1
vote
1answer
11 views
What is the format of S3 lifecycle configuration?
My Problem
An S3 lifecycle configuration changes object storage classes. A typical usage is periodically moving objects to cheaper storage classes after a specified retention period.
I am trying to ...
0
votes
3answers
133 views
using terraform to download zip file from aws s3
I can't seem to find a documentation that will help me download a zip file from aws s3 to an instance using terraform, can someone help me find a solution to this ?
Thank you.
0
votes
0answers
12 views
Can I configure CloudFront to retrieve from different S3 buckets based on path?
I'm setting up a CloudFront distribution that would ideally serve content from many different S3 buckets. The buckets are split up by year, e.g.:
images-2020
images-2019
images-2018
I'd like to set ...
1
vote
1answer
46 views
How does reduced redundancy storage (RRS) fit in S3 storage
This is a highy theoretical question as I am trying to get familiarized with S3 storage classes. The storage classes page in AWS docs does not mention RRS at all. RRS has its own page in AWS docs. The ...
1
vote
3answers
57 views
Using Amazon S3, why do I have inter-region transfers when replication is turned off?
I have an S3 bucked in the Asia Pacific (Sydney) region and have users from Australia using that bucket.
By far my largest charge is for
AWS Data Transfer APS2-USE1-AWS-Out-Bytes
Asia Pacific (...
1
vote
1answer
77 views
Serving secure static S3 site without www
I'm trying to set up AWS to serve a static S3 site over SSL with a custom domain. I'm so close but I'm missing one piece.
S3 static website (works): xyz.s3-website.us-east-2.amazonaws.com
CloudFront ...
0
votes
0answers
16 views
is it possible to change aws s3 access denied message with nginx 403 forbidden
I am using DigitalOcean Spaces. How do I change the XML access denied page to Nginx 403 forbidden page
1
vote
1answer
77 views
what should the ACL value be when limiting S3 object access to CloudFront only?
I followed https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html
But I cannot figure out what should the ACL on AWS.S3.PutObjectRequest be ...
1
vote
1answer
75 views
Setting up S3 to host 'wp-content' folder
If this is not the right SE to post the question, please kindly direct me to the right one.
I've seen configuration of setting up S3 to host wp-content folder, what are the advantages of doing so? ...
0
votes
1answer
70 views
aws s3 ls --summarize switch is broken when trying to get the size of a specific prefix in a bucket
Am I stupid or is this command just totally broken or unclear in usage
I want to see the size of a specific prefix in a bucket
aws s3 ls --summarize --human-readable --recursive 's3://mybucket/...
0
votes
0answers
53 views
How to move nginx cache contents to another machine
I'm running a nginx server on debian 9 to save some bandwidth for my s3 bucket. It currently holds 300gb of data. In order to save some money I want to move this data to another server. I'm currently ...
2
votes
1answer
73 views
Ansible Download file from S3 to ec2 instance Error
I'm trying to learn Ansible. I'm working on creating an instance and uploading a file into it, the file i want to put into the ec2 instance is in stored in S3, but it keeps saying that the destination ...
0
votes
1answer
252 views
Deleting S3 files with a given prefix only
We have a bucket with more than 500,000 objects in it.
I'm assigned a job where I've to delete files which have a specific prefix. There are around 300,000 files with the given prefix in the bucket.
...
0
votes
0answers
32 views
VPCEndpoint s3 access through REST giving access denied
I am trying to create reverse proxy to nginx to S3 with VPCEndpoint.
when I bring up EC2 for Nginx and use aws sdk or cli it works fine but when I make curl request to s3 object I get "Access Denied".
...
1
vote
1answer
37 views
S3 PUT directly to glacier
This AWS post from 2018 makes it sound like it should be possible to make S3 style PUT requests to create archives in glacier now:
I'm interpreting that to mean that you could use the s3 CLI or sdk ...
0
votes
2answers
60 views
Why AWS encryption for EBS and S3 is disabled by default? [closed]
For EBS and S3 encryption (even via the default key from KMS) is disabled by default, as I understood from the documentation, there is no difference to the user if volume/objects is/are encrypted ...
0
votes
2answers
437 views
What is the way to access private S3 bucket files using nginx as proxy?
I have a website and an S3 bucket with numerous images, which should not be accessible directly from any machine with a direct URL.
Those images will be shown on different website pages for ONLY ...
0
votes
1answer
47 views
How can you set up secure Terraform state storage for different infrastructure layers using S3 for backend?
Context
After reading a lot about Terraform and playing with it in minor projects, I'd like to start using it in a real, production environment.
As the environment is mostly in AWS, I'd go for the ...
0
votes
1answer
58 views
How to use SES as an outbound relay for Postfix
I'm looking for a way to both receive emails and then store them in s3 at the same time while using ses, i found out that it can't be done unless i do some way of forwarding, so, what i did was that i ...
1
vote
0answers
22 views
domain masking for a cloudfront distribution
I have a domain test.com which is linked to a cloudfront distribution which then serves static content from s3 bucket. Now I want another domain www.test2.com to masked redirect to test.com. I created ...
0
votes
1answer
60 views
Backing up an SQL backup file from EC2 to Glacier
I have an EC2 server running Sendy (bulk mailing) and I wanted to take backups of the SQL db. Currently, I can download the backup.sql file manually via Virtualmin but I wanted to automate this.
I've ...
2
votes
1answer
104 views
Trying to receive emails AND store them into an S3 bucket
I'm trying to store emails i receive into an s3 bucket, i followed this tutorial and multiple others :
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/receiving-email-getting-started.html
My MX ...
1
vote
1answer
194 views
Flask app on ECS that pull images from s3 bucket
I have a flask app that I want to pull images to display from an s3 bucket.
How do I implement the credentials parts? what I mean is:
Do I use boto3 with aws credentials to pull the images from the ...
0
votes
0answers
16 views
Can I create a lifecycle rule to AWS S3 that moves objects that are already in GLACIER class to DEEP_ARCHIVE after 180 days?
My company has probably close to a petabyte of data in s3 and we've been using S3's GLACIER class to get some cost savings. The files that are already in glacier were manually chosen to be put there ...
1
vote
1answer
282 views
Using aws cli sync include and exclude with delete option
aws s3 sync /WordProcessing/DOCUMENTS s3://mybigbucket --delete --include "*" --exclude ".DS_Store"
If somehow .DS_Store made its way to the AWS S3 mybigbucket (from a previous backup), using the ...
-1
votes
3answers
126 views
aws life cycle rules to transition files from s3 bucket to Glacier storage class
I am creating life cycle rules to move files from bucket to the glacier storage class.
I was wondering if there is an option to copy files from S3 bucket to Glacier using either CLI or console?
I ...
1
vote
1answer
376 views
Does AWS cli do a data integrity check on sync with s3?
I've been looking into using AWS cli for data integrity checks to verify a backup has been transferred from a Linux file server correctly to AWS s3. Likewise, I would like to verify when restoring a ...
