Questions tagged [amazon-s3]

This is for questions on Amazon's cloud storage service.

Filter by
Sorted by
Tagged with
0 votes
0 answers
17 views

How can I see how many bytes does bucket versioning take in an S3 bucket?

How can I see how many bytes does bucket versioning take in an S3 bucket? I tried: aws s3 ls --summarize --human-readable --recursive s3://my-bucket/ but it doesn't show the bucket versioning size.
Franck Dernoncourt's user avatar
1 vote
1 answer
36 views

How I can use the AWS CLI to see whether an S3 bucket has versioning activated?

How I can use the AWS CLI (aws s3) to see whether an S3 bucket has versioning activated?
Franck Dernoncourt's user avatar
0 votes
0 answers
82 views

Why does `aws s3 ls s3://mybucket/ --recursive | wc -l` list fewer files than the number of objects mentioned in the AWS web UI in my S3 bucket?

I have an AWS S3 bucket s3://mybucket/. Running the following command to count all files: aws s3 ls s3://mybucket/ --recursive | wc -l outputs: 279847 Meanwhile, the AWS console web UI clearly ...
Franck Dernoncourt's user avatar
2 votes
2 answers
433 views

How can I know which AWS S3 bucket(s) an AWS key and secret key that can access?

I have an AWS key and secret key that can access some AWS S3 buckets. How can I know which ones?
Franck Dernoncourt's user avatar
-1 votes
1 answer
43 views

How risky is it not to replace the checksum function when copying data between S3 buckets via the AWS web console?

When copying data between S3 buckets via the AWS web console, one may replace the default checksum function: How risky is it not to replace the checksum function when copying data between S3 buckets ...
Franck Dernoncourt's user avatar
1 vote
2 answers
160 views

How can I check if a transfer initiated with `aws s3 cp` was 100% successful?

I've uploaded a large number of files and folders to an AWS S3 bucket via: aws s3 cp . s3://mybucket --recursive How can I check if the transfer was 100% successful?
Franck Dernoncourt's user avatar
0 votes
1 answer
19 views

Archive files in EC2 Instance

I want to create the archives of the files in EC2 instance to load them into the S3 bucket, but I am not getting a proper command to run on the AWS cli.
Shalini Sharma's user avatar
3 votes
3 answers
273 views

How can one copy an S3 bucket as faithfully as possible (e.g., copying all permission settings)?

I want to copy an S3 bucket as faithfully as possible (e.g., copying all permission settings). The standard: aws s3 sync s3://old-bucket s3://new-bucket --source-region us-west-1 --region us-west-2 ...
Franck Dernoncourt's user avatar
1 vote
2 answers
137 views

How can I efficiently move an S3 bucket to a new AWS region while keeping the same bucket name?

My current plan to move an S3 bucket from AWS region us-west-1 to AWS region us-west-2 while keeping the same bucket name seems highly inefficient as it's copying the entire bucket twice: # Create new ...
Franck Dernoncourt's user avatar
0 votes
0 answers
94 views

Velero cannot access same S3 from different EKS cluster

I have velero installed on two different EKS clusters in the same region. All velero installation configurations are same and installed using same charts and credentials, and uses same S3 bucket with ...
Nullpointer's user avatar
0 votes
0 answers
80 views

How to Deploy React App and WordPress on the Same CloudFront Distribution Domain Name with Different Origins and Behaviors?

I'm encountering challenges deploying both a React app and a WordPress site on the same CloudFront Distribution domain name while utilizing different origins and behaviors. Here's my setup: I have a ...
Spiral's user avatar
  • 101
2 votes
1 answer
828 views

Serving HTTPS images with Minio

I followed this documentation, but I can't make my images to be served with HTTPS. I have the following /etc/default/minio configuration file: MINIO_ACCESS_KEY="admin" MINIO_VOLUMES="/...
COil's user avatar
  • 229
0 votes
1 answer
80 views

How to setup a S3 backed Media server with basic auth / password protection

Requirement Serve audios and videos stored in a S3 bucket Selectively protect folders with basic auth (username password); Low latency as much as possible but not necessary Cost is also a factor but ...
Ranjeet Singh's user avatar
0 votes
0 answers
55 views

How to get CloudTrail to record user-data script in the log

I have users who will be able to submit scripts to instances via the aws ec2 run-instances --user-data file://bla.sh option. I have setup CloudTrail to save almost all of the data that I need to log ...
FlippinDemPancakes's user avatar
0 votes
2 answers
310 views

Configuring S3 bucket as alternative storage for Dovecot sdbox

I'm trying to use an S3 bucket for long-term storage for emails, served with Dovecot, with a cron job to doveadm altmove everything older than a few weeks to alternative storage. I've been able to ...
JoshuaS3's user avatar
  • 101
0 votes
1 answer
60 views

Choosing AWS hosting for flutter website

Choosing AWS hosting for flutter website I have a flutter website and i have been thinking to host the site in AWS. I have saw the below features in aws which I can use for hosting my website. AWS ...
Ac. Selvan's user avatar
0 votes
0 answers
95 views

Delete object everyday in S3 bucket

i'm trying to delete folders(objects) in the S3 bucket everyday (with lifecycle rule). I have this scenario : TestBucket -> connect/ -> test/ -> 2023/ -> 12/ -> 01/, 02/, 03/ etc. I ...
Andrea Vitangeli's user avatar
0 votes
1 answer
502 views

My Cloudfront distribution is showing access denied despite enabling all public access to the bucket objects

I am using the following CF policy in my S3 bucket: { "Version": "2008-10-17", "Id": "PolicyForCloudFrontPrivateContent", "...
Navdeep Singh's user avatar
0 votes
1 answer
235 views

S3 site in Cloudfront Preflight 302 Error

I have a static s3 website hosted through CloudFront. In it, I make a request that will result in a redirect if the user is not logged in. Preflight then fails because of a 302: Access to fetch at '...
Demian Licht's user avatar
0 votes
1 answer
310 views

Best method to copy image files from AWS EC2 instance to S3 bucket based on query

What is the most efficient method to copy selected image files from a EC2 instance (Ubuntu 20.04) in production to a S3 bucket while checking if the file exists in EC2? This is a one-time operation. ...
GUIJS's user avatar
  • 1
0 votes
0 answers
133 views

Oracle MySQL operator for kubernetes backup to S3 using service account

I've installed the operator, and trying to create the innodb cluster with backup profile which uses S3. There is no mention of S3 in the documentation, but they mentioned in a conference about S3 ...
Ahmet Cetin's user avatar
1 vote
1 answer
126 views

Is Mastodon remove-orphans safe when multiple instances sharing an S3 bucket?

I have two Mastodon v4.2.0 instances, a staging server and a live/production instance. They share an AWS S3 bucket for file storage. Both are hosted on DigitalOcean. Is the command RAILS_ENV=...
John Skiles Skinner's user avatar
0 votes
0 answers
105 views

Maximize duplicity efficiency

I'm creating incremental backups for a database. The size is around 8 GB per dump. I have a backup creation policy, that looks like this: (don't worry, I already avoided collisions) An increment ...
samoht9277's user avatar
0 votes
0 answers
124 views

Why do I need to map separate s3 buckets for www subdomain requests in AWS?

I have a question about how DNS works in conjunction with AWS s3 buckets that I am trying to understand. I have a modest knowledge about DNS and AWS, but I don't understand why my routing solution ...
Roger Hill's user avatar
0 votes
0 answers
336 views

Dropzone: How to upload video to S3 using presigned URL and play it in browser? Doesn't quite work

I have Dropzone set up to allow users to upload videos directly to an S3 presigned URL using the PUT method. Here is my Dropzone script: var presign = "#"; Dropzone.autoDiscover = false; var ...
peppy's user avatar
  • 73
0 votes
0 answers
391 views

minio A header you provided implies functionality that is not implemented

I'm trying to move object on MinIO yet running into weird issue (even though I'm able to move other objects fine without any issues) summary: mc: Failed to copy https://FQDN/path1/file.txt. A header ...
alexus's user avatar
  • 13.4k
1 vote
1 answer
188 views

Scheduled task to upload a file to S3 with PowerShell ignores user profile settings

I am using Windows task scheduler with a user that has an AWS profile configuration under C:\Users<username>.aws\configuration (content below) [default] s3 = max_concurrent_requests = 300 ...
user2629636's user avatar
0 votes
1 answer
405 views

How to know if an S3 bucket with Cross-Region Replication is fully replicated?

I want to setup my S3 bucket to replicate in another region. Once it's fully replicated, I will make that bucket the main one (the idea is to migrate all data from one region to another). I know how ...
laurent's user avatar
  • 189
0 votes
1 answer
153 views

Proper Setup for a simple AWS S3 website using CloudFront & Route, with base domain and www subdomain

Which is the better design, and why? (see 2 tables) 1. Here is the configuration of my working website (example.com is the prevent domain) as setup via a Youtube Tutorial... Route53 Cloudfront Dist ...
TheArchitecta's user avatar
0 votes
0 answers
116 views

Multi-region internal, domain resolved static website hosted in S3

Statement I do have a website that has a single page application hosted in S3, and a web service hosted in AWS Lambda. To avoid cross-site requests, both share the same domain, and I route the ...
Luís Guilherme's user avatar
0 votes
1 answer
536 views

How to set correct Content-Type HTTP header for all files uploaded on S3 bucket?

What is please the correct way to set Content-Type HTTP header for files uploaded on S3 bucket? We use S3 bucket as an origin server for hosting static websites. Bucket is accessed through the ...
STeN's user avatar
  • 1
0 votes
0 answers
139 views

Enable MFA S3 delete

I have THALES TOTP hardware tokens (MFA) which I would like to use as and additional protection against the accidental S3 object deletion (https://docs.aws.amazon.com/AmazonS3/latest/userguide/...
STeN's user avatar
  • 1
1 vote
2 answers
2k views

ansible - couldn't resolve module/action 'amazon.aws.s3_object' / The module amazon.aws.s3_object was not found in configured module paths

Per requirements, I did installed all the required packages, however amazon.aws.s3_object is one of those things: couldn't be resolved/found was not found in configured module paths step(s), I used ...
alexus's user avatar
  • 13.4k
0 votes
0 answers
93 views

AWS AppStream is unable to push session script logs to s3

I have configured session scripts to log stdout and stderr to s3 bucket as described in https://docs.aws.amazon.com/appstream2/latest/developerguide/use-session-scripts.html#enable-S3-bucket-storage-...
MPP's user avatar
  • 1
0 votes
1 answer
116 views

Best practice for storing personal information on AWS that we shouldn't be able to see

We store customer information that we shouldn't have access to, and don't want to inconvenience users by making them lose information if they forget their passwords, is there a good way to solve this ...
ze234we23's user avatar
0 votes
0 answers
144 views

AWS Lambda's indirect S3 requests

I noticed AWS Lambda causes S3 tier 1 and 2 requests. Do these cost anything after 12 months free tier expires? Specifically: Create new AWS Lambda function in AWS console causes 1 tier 1 S3 request ...
fejyesynb's user avatar
  • 121
0 votes
1 answer
667 views

Restoring Terraform state from backend

My Terraform Git repository ended-up having some unreconciliable data error and I couldn't push a recent change. I exported my commit to a patch, recloned, applied the patch, and then realized I just ...
Dustin Oprea's user avatar
0 votes
0 answers
411 views

botocore.exceptions.ClientError: An error occurred (InvalidToken) when calling the ListObjectsV2 operation

I have an mlflow server that is hosted on aws, that uses s3 in the background for storage. I am trying to run a lambda-function that runs a python script which accesses the mlflow server, loads a ...
ptushev's user avatar
0 votes
2 answers
316 views

How I can Log what and who can access and where my S3 bucket?

Recently I received an email in order to tell me that some IE connections has access to my bucket: eu-central-1|media.myapp | REST.GET.OBJECT|TLSv1|9|[Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1;...
Dimitrios Desyllas's user avatar
0 votes
0 answers
254 views

How do I setup a Google Cloud Storage bucket to be accessible via the Simple Storage Service (S3) standard?

I understand from Google documentation that it's possible for google cloud storage to be accessed via the Simple Storage Service, which is just what I need for a program that supports this. The ...
AdamJones's user avatar
  • 113
0 votes
1 answer
207 views

Storing 100 million files in the same "directory" under S3-compatible storage?

I have > 100 million image files (book covers) as a flat list of files under a single "directory": /images/000000093e7d1825b346e9fc01387c7e449e1ed7 /images/...
BenMorel's user avatar
  • 4,607
0 votes
1 answer
388 views

S3 / Cloudfront / Load balancer Architecture explanation needed

My architecture is like so: I have a static website xyz.com hosted on s3 and served with cloudfront. The website is accessing an api that seats behind a load balancer. I want to make sure that only ...
April Henig's user avatar
-2 votes
1 answer
199 views

Does Amazon have a web server named 'AmazonS3'?

Note that I understand what "Amazon S3" is; I'm asking only about the software that handles the HTTP interface. If I check the HTTP headers for github.com, for example, one of the response ...
QF0's user avatar
  • 213
0 votes
0 answers
1k views

Whitelist an S3 Bucket in server firewall

I am looking for a way to whitelist an S3 Bucket on my client's server. In this S3 bucket there is a public website, thus, my idea is to find the ip address used for this S3 bucket. However, I have ...
VitoCK's user avatar
  • 103
0 votes
1 answer
966 views

Getting 403 (Forbidden) when try to load an S3 file from AWS CloudFront behaviour

I have created an S3 bucket. All objects are publicly accessible. I can access the image file via S3 url: https://S3_BUCKET_NAME.s3.eu-west-1.amazonaws.com/xyz.jpg I can access the image file via ...
tom's user avatar
  • 1
0 votes
1 answer
88 views

Using S3 instead of MinIO for video management

I am relatively new to AWS so I wanted to reach out to the gurus in the community to get some advice on architecture, and I thought I would share my thoughts. I am building out cloud infrastructure ...
Jeremy Leys's user avatar
0 votes
1 answer
44 views

Cloud File Storage: Does zipping files reduce data transmission speed / costs?

If I need to store, say, a 5gb file on a cloud file storage location (e.g., aws s3 bucket), will zipping the file reduce the cost / latency when people download it? Obviously zipping a file reduces ...
Matt's user avatar
  • 105
1 vote
1 answer
556 views

AWS IAM: deny users from creating policies on specific resources

I want to be able to give my admin users the permission to create policies in IAM, but I want to make sure that they aren't able to create a policy that affects a specific resource. To be more ...
Jamie Forrest's user avatar
0 votes
0 answers
29 views

Is storing data in S3 cheaper than paying for cross-az data transfer?

Imagine I have two EC2 instances A and B, in the same region/vpc, but in different AZs. I want to transfer 1TB from A to B. Now let's imagine two scenarios: I send the data from A to B directly. I ...
foo's user avatar
  • 170
1 vote
0 answers
458 views

Amazon S3 life cycle rule for sub folder contents?

With this reference link My s3 bucket name say testbucket. If I create prefix for a rule say. subfolder1/ All the objects in it are applied this rule. But it is not getting applied for objects inside ...
user2331760's user avatar

1
2 3 4 5
23