Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
1,136
questions
1
vote
0
answers
156
views
How do I block TCP ACK causing retransmissions
I am trying to replicate a issue when making requests to S3 that will lead to tcp_retries2 leading to 900 seconds of retransmission attempts before giving up. How would I do this? Is it sufficient to ...
- 11
0
votes
1
answer
18
views
Setting up an architecture for analyses flows of time stream data from multiple sources [closed]
What would be the best practice in terms of AWS for the following:
Many IOT medical devices gather data at around 256kBps
The data is a time series data (a matrix of [Channels X Samples], there can ...
- 1
0
votes
2
answers
1k
views
Unable to mount a folder in Ubuntu to as AWS S3 Bucket
I am attempting to mount a folder on a fresh AWS EC2 instance running Ubuntu 18.04, to AWS S3.
I've been following instructions I found at https://cloud.netapp.com/blog/amazon-s3-as-a-file-system and ...
- 101
1
vote
1
answer
915
views
Will aws s3 be still accessible using pre-signed urls if we create a vpc endpoint gateway?
I have a need to add a vpc endpoint for s3 so i can access it on lambda, but when i try to create the endpoint I get a warning
Warning
When you use an endpoint, the source IP addresses from your ...
- 109
1
vote
1
answer
1k
views
S3 life cycle policy - move to SIA only if size bigger than 128KB
Usecase description:
I'm planning to set a life cycle policiy to move objects from frequent access to infrequent access automatically after 30 days. but in the bucket, there are some small files that ...
0
votes
2
answers
2k
views
Terraform: "S3:path_to_terraform.tfstate": x509: cannot validate certificate for <IP> because it doesn't contain any IP SANs
I'm using s3 to save my state. since this is a custom S3 (and not AWS) I configured it as shown below (using IP + Port).
when running terraform init I received this error:
Successfully configured the ...
- 103
0
votes
0
answers
19
views
AWS S3 bucket policy does not define Put, yet it is allowed [duplicate]
I have a public static website which images can be uploaded to. For this I enabled static website hosting and set Block public access to OFF. I then added this policy
{
"Version": "...
- 133
0
votes
0
answers
2k
views
AWS S3 bucket policy for get and upload
I have two buckets each with their own goals. So far I cannot understand the complicated configuration of bucket settings. It seems there are three things to configure
Block public access
Bucket ...
- 133
2
votes
2
answers
1k
views
AWS S3 Static Website vs CloudFront OIA
It seems there are kind of two ways to host a static website on S3/CloudFront.
Create S3 bucket with full public access and Static Website set to Enabled + CloudFront config
Create S3 bucket with no ...
- 133
0
votes
1
answer
384
views
API gateway cache every hour
My API is set up as follows:
API Gateway > Lambda function > S3
Every hour the S3 content is updated at the same time. How do I configure API gateway to cache the endpoint's responses but serve ...
- 107
0
votes
1
answer
587
views
how to download archive in aws s3 glacier
first, sorry for my bad english.
i create a vault in glacier service and then i uploaded an archive into vault and in out-put, aws gave back me a archive id. then using the archive-id, i created a job ...
- 109
0
votes
0
answers
71
views
AWS/S3/Cloudfront/GoDadday and subdomains
I am wondering and hoping this is possible.
On AWS I have two s3 buckets to separate a client facing application and an administrative application
mysite-app
mysite-admin
I have a cloudfront ...
- 133
0
votes
1
answer
337
views
Transition of a large bucket to S3 Glacier
I want to move a 72 TB size bucket from S3 Standard to S3 Glacier.
I'm going to create a Lifecycle rule and set "Transition current versions of objects between storage classes" to 1 day.
I ...
- 31
0
votes
2
answers
260
views
Best Practice for AWS IAM access keys for use with AWS SDK
I want to know the best practice used by big company's for programmatic access for multiple AWS services as there are multiple programs needing access to different-2 services so how it is managed? Did ...
- 49
0
votes
0
answers
847
views
400 Bad Request errors (infrequent) on public Amazon S3 assets
We are hosting S3 public assets (images) under a local path using a reverse proxy from NGINX to S3.
We have noticed periodic errors in our logs (400 errors) which are very infrequent, but are causing ...
- 715
0
votes
0
answers
286
views
AWS S3 static website: unable to access https://mydomain.tld but http://mydomain.tld redirects to https correctly
I've set up a static website on AWS S3, using a custom domain hosted at Namecheap, and Cloudfront serving over https. I am able to access my site at http://mydomain.tld, which redirects (using a DNS ...
- 101
1
vote
0
answers
368
views
S3 through Nginx Proxy doesn't allow more than 10 files to be downloaded simultaneously
I setup a Nginx Proxy in front of my S3 object storage and I'm able to download files without any issues so far.
However, if I try to download the same file from, let's say, 30 different servers, at ...
- 111
0
votes
1
answer
2k
views
Why can't my DB back up to my S3 bucket?
I'm trying to test a differential backup and restore on an RDS SQL Server 2019 instance in preparation for migrating some web resources into AWS. Both the EC2 instance (Windows Server 2019) on which I'...
- 153
0
votes
1
answer
908
views
cannot Delete objects on s3 ceph bucket
I have a ceph cluster and running a few s3 bucket on it, 'gitlab-s3-api' user has full permission on everything on ( users=;buckets=;metadata=;usage=;zone=* ) but very oddly it cannot delete any file ...
1
vote
1
answer
4k
views
Domain to S3 Bucket AccessDenied
Hello i have a problem i recibe AccessDenied in my domain
I upload my site in to a AWS S3 Bucket, i make public, create a static page, but when i access via my domain.com/mypage i get a AccessDenied.
...
- 11
0
votes
1
answer
193
views
Cloudfront distribution with react app and wordpress app hosted on same domain - how to get browsers with react app cached to render wordpress app?
I have a domain that hosts a react website in an s3 bucket via cloudfront. There is also a wordpress site hosted on a subdomain of that domain, and in the cloudfront distribution for the web app, I ...
- 103
6
votes
1
answer
4k
views
AWS: How to redirect many domains to a page on another domain?
My objective
I have a number of domains (e.g. 10 or 20) and I would like to redirect any visitors to anywhere on those pages to one page on another domain (for example my stackoverflow.com profile ...
- 1,189
1
vote
1
answer
2k
views
AWS Cloudfront CORS trouble with font files
I have a Javascript application that is delivered to the browser from a CloudFront endpoint. All static files of the application are hosted in an S3 bucket. That involves a whole bunch of javascript ...
- 247
3
votes
2
answers
10k
views
How to *actually* exclude a directory in AWS S3 sync?
The aws s3 sync command has an --exclude flag which lets you exclude a folder from the sync. However, even though the files are not uploaded from that directory, the command still looks at and ...
- 171
0
votes
0
answers
224
views
AWS SES Managing and Forwarding Incoming Mail
I'm probably in over my head with this one. Please bare with me while i fumble through my explanation of what it is that i am trying to accomplish here.
We own a domain name. Have access to the back ...
- 1
1
vote
0
answers
3k
views
Converting specific folder of S3 into browsable directory list without making it public
I have a bucket that I'd like to access using a browser similar to http://data.openspending.org/
and I'd like only a subfolder to be visible.
So if Bucket1 has multiple folders, I only wanna show and ...
- 113
3
votes
1
answer
3k
views
Why is AWS S3 Object count is measured iin units used to measure file size?
In AWS, the files/objects are stored in S3 bucket and the files/object count should list the total no of files/objects stored in S3 bucket as an INTEGER value.
But I see the S3 object count as ...
- 155
3
votes
1
answer
764
views
AWS S3 File Browser for Windows
Is there any official AWS GUI like S3 file explorer for Windows?
I know there's plenty non-official like S3 Browser but, I am avoiding using third-party software.
- 1,351
0
votes
1
answer
2k
views
Why is my database restore failing with ConnectFailure exception?
I am trying to restore a SQL Server database to an AWS RDS instance from a full backup file created on a different non-RDS server. I am running the operation from an EC2 Windows Server instance in the ...
- 153
1
vote
1
answer
1k
views
AWS S3 Policy: One non-public bucket, separate sub-folders for each user, restricted access
at the moment I'm struggling how to create a secure policy for my Amazon S3 bucket.
My plan is to have one bucket with several sub-folders for separate (IAM) users.
Access should only be ...
1
vote
0
answers
89
views
AWS Elastic Beanstalk SNI SSL in S3
In our application we need to support customer ssl certificates and domains.
Application stack: AWS Elastic Beanstalk, NodeJs and nginx server.
I've found that we can use nginx variable "$...
- 11
0
votes
1
answer
889
views
aws s3 mv command is not deleting the folder structure
aws s3 mv s3://test/paltest/ . --recursive
All files within the paltest is been moved but paltest is not deleted. But i am able to remove paltest using aws s3 rm s3://test/paltest/
0
votes
1
answer
321
views
AWS Glacier and Ransomware
I'm trying to understand the structure of how AWS Glacier works because I have a problem.
Problem:
I have a NAS that backs up to Glacier about once a week. About two weeks ago the NAS got infected ...
1
vote
0
answers
256
views
Use increment instead of random string in AWS S3
When I upload files with similar filenames Amazon S3 adds random chars to the filename. How I can change it to use increment instead?
1
vote
0
answers
394
views
Should I archive file copies from S3 into Glacier one-by-one Or should I zip them first?
I have an s3 bucket named media and in it I have a folder named contracts that stores pdf files ~400kb each. For the files in this folder I keep a record in a database as well with the following ...
0
votes
1
answer
102
views
Using IAM Users for customer access to a bucket
For each of my customers, they have data stored on individual dropbox accounts and im in the process of consolidating all of these into AWS in the shape of a single s3 bucket where each customer has ...
- 101
0
votes
1
answer
320
views
Cannot write to AWS S3 bucket using CLI
Im running a Github CD actions that takes some files from my repo and tries to upload them to a S3 bucket I have.
The action runs this command:
/usr/bin/docker run --name ...
- 375
-1
votes
1
answer
88
views
How to get complete bucket access of aws s3 as public?
When we set a file in s3 bucket as public, we can directly download it from its URL.
But, if I set a bucket or a sub folder in bucket to public, I am getting below error.
<Error>
<Code>...
- 392
0
votes
0
answers
216
views
Copy old data from local Centos to AWS S3 bucket
I have the following task to complete. I hope someone can provide the feedback to get me started.
We have a directory with thousands of images in many folders and sub-folders
The goal is to have a ...
-1
votes
2
answers
422
views
How do I share all resources between two AWS Root accounts?
We have one person that has been in charge of all AWS resources for our company, including EC2, elastic IPs, EBS storage, etc. Now we want a second person to also have access to ALL the resources in ...
- 109
1
vote
1
answer
20k
views
AWS S3 bucket - Allow download files to every IAM and Users from specific AWS Account
Look for a policy for S3 bucket that will allow all IAM roles and users from different account, to be able to download files from the bucket that is located in my AWS account.
Thanks for help
- 141
3
votes
1
answer
798
views
Delete AWS RDS S3 Exported Snapshots
So I have created S3 Exports from existing snapshots in RDS. I would like to delete duplicated ones. But I don't see any button to delete them.
I have already deleted the files from the bucket. But ...
- 131
1
vote
0
answers
204
views
Combination of S3 bucket, fuse, rclone mount and incron - not detecting changes
and thanks for reading.....
I'm trying to setup a method whereby I have an on-premise Centos 7 server which uses rclone (mount) to mount an AWS S3 bucket as a file system. I've installed fuse to make ...
- 11
20
votes
2
answers
28k
views
Why don't I have permissions to edit an S3 bucket policy when logged on as the person who created the AWS account
I am new to AWS. I am setting up an S3 bucket that I want to use to store media files for a Django App I am developing.
I am logged in as the person who created the AWS account, but when I click on ...
- 615
1
vote
0
answers
131
views
Fastest way to import files in Spark?
I’m playing around with Spark 3.0.1 and I’m really impressed by the performance with Spark SQL on GB of data.
I’m trying to understand what’s the best way to import multiple JSON files in the Spark ...
- 193
1
vote
1
answer
1k
views
Unable to access S3 static website using https
I have followed all the steps from https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3/ but my website still cannot be accessed using https.
My certificate was issued ...
- 83
-1
votes
1
answer
125
views
Suggestions for AWS setup wanted
For my next project, I'm giving AWS and server-less a go, but I'm not sure how to reason about the different AWS services. So here's hoping that someone here has a good feel for it.
The project ...
0
votes
1
answer
305
views
Split Traffic Between S3 Buckets
I have a static S3 site in bucket_old and will rebuild it route by route in S3 bucket_new using the strangler pattern.
URLs /home, /contact map to bucket_new
All other URLs map to bucket_old
What ...
- 101
0
votes
1
answer
536
views
Cloudfront SSL with Subdomain pointing to S3 bucket works, but still redirects to S3 URL
I've set up an S3 Bucket that I want to use as a CDN but with my own subdomain.
For example, myfiles.mydomain.com.
So, I set up a Cloudfront web distribution that points to the bucket named "...
- 247
0
votes
1
answer
3k
views
How to access an aws s3 bucket using Access Key ID and secret access Key?
I have a free account on AWS and somebody shared me an Access key ID and Secret access key to access their s3 bucket.
I am new in AWS, how can I access their s3 bucket using AWS web console?
- 101