2022 Developer Survey is open! Take survey.

Questions tagged [amazon-s3]

This is for questions on Amazon's cloud storage service.

Filter by
Sorted by
Tagged with
0 votes
0 answers
31 views

Unable to access few objects in S3

Why some objects are accessible in S3 bucket and some are not. Following are the below errors we faced. Though I checked the access and it seems to be fine. Exception while connecting S3: An error ...
user avatar
  • 1
0 votes
1 answer
20 views

How do I extract a list of all folders of my AWS S3 storage bucket or directories?

Does Amazon provide an easy way extract a list of all folders that have files greater than 500 MB from a s3 bucket? want to limit the scope to the '/files/ftp_upload/' directories also This is so I ...
user avatar
0 votes
1 answer
32 views

AWS Policy to Read/write RDS

In my scenario , I want a policy that will allow reading and writing of abc-database-backups/rds/postgresql-backup on S3? We'll want the my servers to have that access added. Is creating a role and ...
user avatar
0 votes
1 answer
29 views

How do I open a private static website using boto3?

I have a static website consisting of multiple HTML, CSS, and media files uploaded to a private S3 bucket. I would like to access the website using my web browser. I'm currently using boto3 to ...
user avatar
0 votes
1 answer
47 views

Trying to understand S3 Lifecycle rule

I am working on an e-commerce website. I store product photos in an s3 bucket. Once the product is deleted, I also delete the photos from s3 bucket. I have S3 bucket versioning enabled. I am not ...
user avatar
1 vote
1 answer
124 views

What is the difference between S3 Multi-Region Access Points and AWS CloudFront?

Can anyone explain the difference between S3 Multi-Region Access Points and AWS CloudFront? The documentation for a multi-region access point states: Multi-Region Access Points offer a global S3 ...
user avatar
0 votes
1 answer
287 views

How to download millions of s3 files and compress them on the fly?

I have an S3 bucket with millions of files, and I want to download all of them. Since I don't have enough storage, I would like to download them, compress them on the fly and only then save them. How ...
user avatar
  • 1
1 vote
0 answers
255 views

CloudFront CORS request using signed cookies and withCredentials, not sending back Access-Control-Allow-Credentials unless I include some extra header

I'm having a very strange issue that I can't seem to crack. I configured a private CloudFront distribution to serve content from a private S3 bucket. I am using signed cookies to grant access to the ...
user avatar
  • 61
0 votes
1 answer
53 views

Bandwidth consumption of Amazon S3 bucket mounted with S3fs on a VPS

I have : VPS server on Vultr with web server and Wordpress installed. Uploads folder is mounted on Amazon S3 bucket using s3fs. If a user downloads a 1GB file from my website will the traffic be ...
user avatar
1 vote
1 answer
29 views

S3 policy to allow by IP can't be changed

We have an S3 bucket with website content in it that's served to the public through a proxy cache. The access to the bucket is limited to the IPs used by the proxy. This is all working fine. New IPs ...
user avatar
0 votes
1 answer
96 views

S3 Logs event Issue

Is there a way to see what actions the 'g2' IAM user is performing in S3, and which IP(s) they are running from? I have already enabled the logging of S3 actions. One point I’m still not able to ...
user avatar
0 votes
1 answer
19 views

Sending push notification to s3 origin website from lambda

I want to send a websockets notification from lambda function to clients, on a website hosted via s3 origin with cloudfront. Could someone help me with this, please? Thanks.
user avatar
  • 3
0 votes
1 answer
22 views

aws cloudfront custom ssl pricing for subdomain and www

I have created two s3 origin website linked with cloudfront and used acm certificate. But I recently found out pricing are different between https://admin.example.com and https://www.example.com. www....
user avatar
  • 3
0 votes
0 answers
56 views

CloudFront delivery is slow in particular location

In our scenario, I found that our CloudFront delivery particularly in the Japan area is very slow for a few days. which lead our site to become slow in the time of loading the media contents .our s3 ...
user avatar
1 vote
0 answers
22 views

Calculation of S3’s byte-hour

S3 uses the concept of Byte-Hour in billing: https://aws.amazon.com/s3/faqs/#Billing Unfortunately (in typical Amazon AWS style) they left some details unsaid, namely what happens in the case of ...
user avatar
0 votes
1 answer
32 views

DNS - GoDaddy for mail and AWS for rest

I have a GoDaddy hosting account but for some of my websites I would like to host them in AWS S3. My question is how to set DNS entries in Route 53 so that I keep web traffic served from S3 but I ...
user avatar
1 vote
0 answers
32 views

How do I block TCP ACK causing retransmissions

I am trying to replicate a issue when making requests to S3 that will lead to tcp_retries2 leading to 900 seconds of retransmission attempts before giving up. How would I do this? Is it sufficient to ...
user avatar
0 votes
1 answer
14 views

Setting up an architecture for analyses flows of time stream data from multiple sources [closed]

What would be the best practice in terms of AWS for the following: Many IOT medical devices gather data at around 256kBps The data is a time series data (a matrix of [Channels X Samples], there can ...
user avatar
  • 1
0 votes
1 answer
150 views

Unable to mount a folder in Ubuntu to as AWS S3 Bucket

I am attempting to mount a folder on a fresh AWS EC2 instance running Ubuntu 18.04, to AWS S3. I've been following instructions I found at https://cloud.netapp.com/blog/amazon-s3-as-a-file-system and ...
user avatar
0 votes
0 answers
138 views

How to Bulk Delete 1.4 Million AWS S3 Access Logs

We hadn't specified a target bucket for our AWS access logs & I believe we set up an infinite loop https://aws.amazon.com/premiumsupport/knowledge-center/s3-server-access-logs-same-bucket leading ...
user avatar
1 vote
1 answer
219 views

Will aws s3 be still accessible using pre-signed urls if we create a vpc endpoint gateway?

I have a need to add a vpc endpoint for s3 so i can access it on lambda, but when i try to create the endpoint I get a warning Warning When you use an endpoint, the source IP addresses from your ...
user avatar
  • 109
0 votes
1 answer
80 views

S3 life cycle policy - move to SIA only if size bigger than 128KB

Usecase description: I'm planning to set a life cycle policiy to move objects from frequent access to infrequent access automatically after 30 days. but in the bucket, there are some small files that ...
user avatar
0 votes
2 answers
291 views

Terraform: "S3:path_to_terraform.tfstate": x509: cannot validate certificate for <IP> because it doesn't contain any IP SANs

I'm using s3 to save my state. since this is a custom S3 (and not AWS) I configured it as shown below (using IP + Port). when running terraform init I received this error: Successfully configured the ...
user avatar
  • 103
0 votes
0 answers
19 views

AWS S3 bucket policy does not define Put, yet it is allowed [duplicate]

I have a public static website which images can be uploaded to. For this I enabled static website hosting and set Block public access to OFF. I then added this policy { "Version": "...
user avatar
  • 113
0 votes
0 answers
177 views

AWS S3 bucket policy for get and upload

I have two buckets each with their own goals. So far I cannot understand the complicated configuration of bucket settings. It seems there are three things to configure Block public access Bucket ...
user avatar
  • 113
0 votes
2 answers
189 views

AWS S3 Static Website vs CloudFront OIA

It seems there are kind of two ways to host a static website on S3/CloudFront. Create S3 bucket with full public access and Static Website set to Enabled + CloudFront config Create S3 bucket with no ...
user avatar
  • 113
0 votes
0 answers
38 views

Do I have to set up S3 for Laravel application on Elastic Beanstalk?

I have set up an Elastic Beanstalk environment on top of PHP platform and deployed Laravel code. I have realized that each time I deploy code to EB environment, the storage becomes empty (naturally ...
user avatar
  • 101
0 votes
1 answer
102 views

API gateway cache every hour

My API is set up as follows: API Gateway > Lambda function > S3 Every hour the S3 content is updated at the same time. How do I configure API gateway to cache the endpoint's responses but serve ...
user avatar
  • 107
0 votes
1 answer
231 views

how to download archive in aws s3 glacier

first, sorry for my bad english. i create a vault in glacier service and then i uploaded an archive into vault and in out-put, aws gave back me a archive id. then using the archive-id, i created a job ...
user avatar
  • 69
0 votes
0 answers
48 views

Issue with syncing data from Amazon RDS to Amazon S3

I am trying to sync binlogs in RDS mysql DB with Amazon S3 in real time to query using Amazon Athena. I have tried using debezium as source connector for Apache Kafka connect- kafka stores streaming ...
user avatar
0 votes
0 answers
37 views

AWS/S3/Cloudfront/GoDadday and subdomains

I am wondering and hoping this is possible. On AWS I have two s3 buckets to separate a client facing application and an administrative application mysite-app mysite-admin I have a cloudfront ...
user avatar
  • 113
0 votes
1 answer
47 views

Transition of a large bucket to S3 Glacier

I want to move a 72 TB size bucket from S3 Standard to S3 Glacier. I'm going to create a Lifecycle rule and set "Transition current versions of objects between storage classes" to 1 day. I ...
user avatar
0 votes
2 answers
99 views

Best Practice for AWS IAM access keys for use with AWS SDK

I want to know the best practice used by big company's for programmatic access for multiple AWS services as there are multiple programs needing access to different-2 services so how it is managed? Did ...
user avatar
  • 49
0 votes
0 answers
287 views

400 Bad Request errors (infrequent) on public Amazon S3 assets

We are hosting S3 public assets (images) under a local path using a reverse proxy from NGINX to S3. We have noticed periodic errors in our logs (400 errors) which are very infrequent, but are causing ...
user avatar
  • 685
0 votes
0 answers
64 views

Deny unencrypted s3 buckets via SCP

Folks just wondering if there’s ability to attach a SCP to OU accounts denying S3 buckets from being created if default encryption is not opted upfront. From cloud trail it’s apparent that ...
user avatar
0 votes
0 answers
32 views

AWS S3 static website: unable to access https://mydomain.tld but http://mydomain.tld redirects to https correctly

I've set up a static website on AWS S3, using a custom domain hosted at Namecheap, and Cloudfront serving over https. I am able to access my site at http://mydomain.tld, which redirects (using a DNS ...
user avatar
  • 101
1 vote
0 answers
129 views

S3 through Nginx Proxy doesn't allow more than 10 files to be downloaded simultaneously

I setup a Nginx Proxy in front of my S3 object storage and I'm able to download files without any issues so far. However, if I try to download the same file from, let's say, 30 different servers, at ...
user avatar
0 votes
1 answer
383 views

Why can't my DB back up to my S3 bucket?

I'm trying to test a differential backup and restore on an RDS SQL Server 2019 instance in preparation for migrating some web resources into AWS. Both the EC2 instance (Windows Server 2019) on which I'...
user avatar
  • 143
0 votes
1 answer
368 views

cannot Delete objects on s3 ceph bucket

I have a ceph cluster and running a few s3 bucket on it, 'gitlab-s3-api' user has full permission on everything on ( users=;buckets=;metadata=;usage=;zone=* ) but very oddly it cannot delete any file ...
user avatar
0 votes
0 answers
29 views

Automate AWS RDS SQL server import from on-premise SQL server

I want to create a full daily backup from an on-premise SQL server, store it on S3, and then restore it onto an existing Amazon RDS SQL Server. Something like this documentation: https://docs.aws....
user avatar
  • 1
0 votes
1 answer
1k views

Domain to S3 Bucket AccessDenied

Hello i have a problem i recibe AccessDenied in my domain I upload my site in to a AWS S3 Bucket, i make public, create a static page, but when i access via my domain.com/mypage i get a AccessDenied. ...
user avatar
  • 1
0 votes
1 answer
39 views

Cloudfront distribution with react app and wordpress app hosted on same domain - how to get browsers with react app cached to render wordpress app?

I have a domain that hosts a react website in an s3 bucket via cloudfront. There is also a wordpress site hosted on a subdomain of that domain, and in the cloudfront distribution for the web app, I ...
user avatar
  • 103
0 votes
0 answers
224 views

Mount S3FS mount to EFS additionally and share to other Kubernetes pod

What I need to achieve that is almost done without the EFS/S3FS share: SFTP pod used by some microservices that process content and deliver back processed content have sftp users that go to tenant ...
user avatar
4 votes
1 answer
1k views

AWS: How to redirect many domains to a page on another domain?

My objective I have a number of domains (e.g. 10 or 20) and I would like to redirect any visitors to anywhere on those pages to one page on another domain (for example my stackoverflow.com profile ...
user avatar
  • 1,110
1 vote
1 answer
652 views

AWS Cloudfront CORS trouble with font files

I have a Javascript application that is delivered to the browser from a CloudFront endpoint. All static files of the application are hosted in an S3 bucket. That involves a whole bunch of javascript ...
user avatar
  • 207
0 votes
0 answers
59 views

AWS S3 links return 403 when application is run using Docker Swarm, but works fine when run with Docker Compose

I have a Django application on EC2 (ubuntu) instance using S3 for static & media files, which is set up currently using Docker Compose. I'm trying to convert it to use Docker Swarm, but in doing ...
user avatar
2 votes
2 answers
3k views

How to *actually* exclude a directory in AWS S3 sync?

The aws s3 sync command has an --exclude flag which lets you exclude a folder from the sync. However, even though the files are not uploaded from that directory, the command still looks at and ...
user avatar
  • 121
0 votes
0 answers
145 views

AWS SES Managing and Forwarding Incoming Mail

I'm probably in over my head with this one. Please bare with me while i fumble through my explanation of what it is that i am trying to accomplish here. We own a domain name. Have access to the back ...
user avatar
  • 1
0 votes
0 answers
169 views

Image resizing and caching in NGINX after authentication and 302 redirect

I'm having issues with resizing and caching images. Here is my non-performant setup without caching: Client makes request for website.com/static/image.jpeg NGINX routes to API API authenticates user, ...
user avatar
0 votes
0 answers
24 views

S3 Security Policy for Bucket while allowing signed urls

ugh, bucket policies. I am trying to write a bucket security policy that would keep an outsider from listing the structure and data in the bucket but at the time my AWS users needs to be able to pull/...
user avatar
  • 109

1
2 3 4 5
22