Questions tagged [amazon-s3]

This is for questions on Amazon's cloud storage service.

286 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1 vote
0 answers
7k views

Intermidiate "upstream prematurely closed connection while reading upstream"

I am getting an intermediate "upstream prematurely closed connection while reading upstream" errors on my server. The error only occurs on the data that is requested throe the server from Amazon S3 ...
Alex Frenkel's user avatar
1 vote
1 answer
562 views

Allow access to S3 bucket via VPC or via console?

I want my S3 bucket to be accessible in two ways: Via the console when I'm logged in, so I can modify files Via EC2 instances in a certain VPC (IAM roles are set up correctly for this) Here's the ...
fredley's user avatar
  • 575
1 vote
2 answers
510 views

AWS/S3 - creating a user (through console) to grant r/w bucket perms to for 1 bucket?

I have an Amazon Web Services S3 bucket I want to use with the android data syncing app 'FolderSync'. Towards that I want to set limited perms on the bucket for a new user. Within the AWS management ...
glaucon's user avatar
  • 253
1 vote
1 answer
843 views

Amazon Cloudfront with S3 bucket - 2 origins

I am bit confused with setting up Amazon Cloudfront and S3 bucket. I've got my bucket setup and using the wordpress WP Offload S3 plugin to store any images in this bucket. When checking the image ...
Joe Bloggs's user avatar
1 vote
0 answers
880 views

Amazon Elastic Beanstalk with File Access to Amazon S3

i've a problem with my Amazon Elastic Beanstalk and Amazon S3 setup. From the EB instance i want to load a file with node (fs.readFileSync), i tried a ton of configurations - but non of them worked, ...
Michael J. Zoidl's user avatar
1 vote
1 answer
2k views

s3cmd error on sync: S3 error: 400 (InvalidRequest)

I'm trying to configure s3cmd to backup some directories on my server to my AWS S3 bucket. My problem is that when I do the sync command: s3cmd sync -r -p --no-delete-removed /path/to/dir s3://...
Bart Bergmans's user avatar
1 vote
0 answers
55 views

fast backup script in linux

I have a server with more applications and do normal shell command to take backups. it took couple of days to complete. Tried to install duplicity. but its told some packages are not found. its Debian ...
Bala Murugan's user avatar
1 vote
0 answers
33 views

Enabling S3 bucket as website using cli

I figured out how to create an S3 bucket using aws command line. Is it possible to enable the s3 bucket I created as website using aws cli? The reason I'm doing this is because we're automating ...
devwannabe's user avatar
1 vote
0 answers
1k views

Bucket policy that respects pre-signed URLs OR IP Address deny?

I would like to be able to restrict access to files in a S3 bucket in multiple ways. This is due to the fact that the files stored can be accessed in different manners. We do this because we have TBs ...
johnmontfx's user avatar
1 vote
0 answers
2k views

Access denied for the request of data from s3 bucket on high load

I am trying to fetch static files from s3 bucket in case of 502 error. This is how I have configured nginx for requests : error_page 502 = @static; location / { index index.php; try_files $uri $...
Satys's user avatar
  • 193
1 vote
0 answers
8k views

AWS security group egress rules for S3

I manually created a new security group using the AWS CLI. I created ingress rules that allow incoming connections only from my company's public IP address using the known ports for SSH (22) and ...
HaroldFinch's user avatar
1 vote
1 answer
2k views

Can I use an IAM role to grant my Heroku app access to my Amazon S3 bucket?

Heroku: Using AWS S3 to Store Static Assets and File Uploads suggests using my AWS security credentials to enable my Heroku app to access my Amazon S3 bucket. However, isn't it better practice (as ...
ma11hew28's user avatar
  • 799
1 vote
1 answer
429 views

AWS S3 bucket permissions

I've created AWS s3 buckets with several thousand objects, initially the access policy was restricted to authenticated users, but later due to some requirement we need to make all objects in this ...
vasco.debian's user avatar
1 vote
1 answer
1k views

jboss cluster in ec2 with s3ping

Am trying to get s3ping discovery method working in wildfly 8.0.0. I am using default Amazon Linux AMI and using chef to launch instances in ec2 based on this tutorial http://aws.typepad.com/...
karthik101's user avatar
1 vote
2 answers
2k views

AWS S3 Sync options duplicates / weird behaviour

I am trying to sync a folder with 2M+ files to S3, everything went OK but then 40,000 files were not uploaded (server crashed randomly), when I tried to do a sync command again, it started from 0, ...
Robert W. Hunter's user avatar
1 vote
1 answer
972 views

Nginx proxy cache invalidating request to s3

I have set up a proxy from my application server to a private s3 bucket to cache requests. I was having some trouble with it where s3 was rejecting my download requests (403 forbidden) and after some ...
Christopher Reid's user avatar
1 vote
0 answers
100 views

svnadmin dump returns error insufficient disk space

I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked. I have Apache Subversion hosted on an ...
Terence's user avatar
  • 11
1 vote
0 answers
455 views

Duplicity -> S3 backup failing with "[Errno 105] No buffer space available"

I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
Greg's user avatar
  • 249
1 vote
1 answer
684 views

Glacier retrieval notification of s3 objects

I have some s3 buckets which send their objects to glacier after x days of creation. When I go to retrieve those objects on the s3 interface, I need to keep reloading page to see when restoration has ...
alexandresaiz's user avatar
1 vote
0 answers
2k views

Amazon S3: allow user access to subfolder only on any bucket

I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket. This is my policy so far (reading works, but it ...
DiegoSalazar's user avatar
1 vote
0 answers
336 views

Handling multiple server blocks in nginx configuration

I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com. Eg:If ...
KillABug's user avatar
  • 127
1 vote
0 answers
579 views

Amazon S3 sync theory

I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
dKen's user avatar
  • 193
1 vote
1 answer
1k views

Can AWS S3 files in buckets get deleted by a download timing out?

A few AWS S3 files might have gotten deleted while trying to download everything for a local backup. The download/connection timed-out a couple times. The next day several HTML/JS files are missing ...
Miles Lukas's user avatar
1 vote
1 answer
897 views

Reducing latency for different geographic regions on Amazon Cloud

I have got an application which has three components Application code : Amazon EC2 US-EAST-1 instance Application images, and other static data : Amazon S3 with CloudFront Application Database : ...
Shoaibi's user avatar
  • 819
1 vote
1 answer
803 views

Why would anyone disable S3 KMS "bucket key" option?

As per the documentation: Amazon S3 Bucket Keys reduce the cost of Amazon S3 server-side encryption using AWS Key Management Service (SSE-KMS). This new bucket-level key for SSE can reduce AWS KMS ...
Rino Bino's user avatar
  • 603
1 vote
1 answer
3k views

Setting cache for already uploaded amazon s3 files

In amazon s3 for our site through s3cmd tool we uploaded around 60GB of files including mp3, jpg to a bucket. But now when we checked site slowness files are not caching and further checking we came ...
GEO PC's user avatar
  • 31
0 votes
1 answer
18 views

Archive files in EC2 Instance

I want to create the archives of the files in EC2 instance to load them into the S3 bucket, but I am not getting a proper command to run on the AWS cli.
Shalini Sharma's user avatar
0 votes
0 answers
47 views

Velero cannot access same S3 from different EKS cluster

I have velero installed on two different EKS clusters in the same region. All velero installation configurations are same and installed using same charts and credentials, and uses same S3 bucket with ...
Nullpointer's user avatar
0 votes
0 answers
72 views

How to Deploy React App and WordPress on the Same CloudFront Distribution Domain Name with Different Origins and Behaviors?

I'm encountering challenges deploying both a React app and a WordPress site on the same CloudFront Distribution domain name while utilizing different origins and behaviors. Here's my setup: I have a ...
Spiral's user avatar
  • 101
0 votes
1 answer
56 views

How to setup a S3 backed Media server with basic auth / password protection

Requirement Serve audios and videos stored in a S3 bucket Selectively protect folders with basic auth (username password); Low latency as much as possible but not necessary Cost is also a factor but ...
Ranjeet Singh's user avatar
0 votes
0 answers
49 views

How to get CloudTrail to record user-data script in the log

I have users who will be able to submit scripts to instances via the aws ec2 run-instances --user-data file://bla.sh option. I have setup CloudTrail to save almost all of the data that I need to log ...
FlippinDemPancakes's user avatar
0 votes
2 answers
282 views

Configuring S3 bucket as alternative storage for Dovecot sdbox

I'm trying to use an S3 bucket for long-term storage for emails, served with Dovecot, with a cron job to doveadm altmove everything older than a few weeks to alternative storage. I've been able to ...
JoshuaS3's user avatar
  • 101
0 votes
1 answer
49 views

Choosing AWS hosting for flutter website

Choosing AWS hosting for flutter website I have a flutter website and i have been thinking to host the site in AWS. I have saw the below features in aws which I can use for hosting my website. AWS ...
Ac. Selvan's user avatar
0 votes
0 answers
72 views

Delete object everyday in S3 bucket

i'm trying to delete folders(objects) in the S3 bucket everyday (with lifecycle rule). I have this scenario : TestBucket -> connect/ -> test/ -> 2023/ -> 12/ -> 01/, 02/, 03/ etc. I ...
Andrea Vitangeli's user avatar
0 votes
1 answer
395 views

My Cloudfront distribution is showing access denied despite enabling all public access to the bucket objects

I am using the following CF policy in my S3 bucket: { "Version": "2008-10-17", "Id": "PolicyForCloudFrontPrivateContent", "...
Navdeep Singh's user avatar
0 votes
1 answer
200 views

S3 site in Cloudfront Preflight 302 Error

I have a static s3 website hosted through CloudFront. In it, I make a request that will result in a redirect if the user is not logged in. Preflight then fails because of a 302: Access to fetch at '...
Demian Licht's user avatar
0 votes
1 answer
281 views

Best method to copy image files from AWS EC2 instance to S3 bucket based on query

What is the most efficient method to copy selected image files from a EC2 instance (Ubuntu 20.04) in production to a S3 bucket while checking if the file exists in EC2? This is a one-time operation. ...
GUIJS's user avatar
  • 1
0 votes
0 answers
121 views

Oracle MySQL operator for kubernetes backup to S3 using service account

I've installed the operator, and trying to create the innodb cluster with backup profile which uses S3. There is no mention of S3 in the documentation, but they mentioned in a conference about S3 ...
Ahmet Cetin's user avatar
0 votes
0 answers
94 views

Maximize duplicity efficiency

I'm creating incremental backups for a database. The size is around 8 GB per dump. I have a backup creation policy, that looks like this: (don't worry, I already avoided collisions) An increment ...
samoht9277's user avatar
0 votes
0 answers
115 views

Why do I need to map separate s3 buckets for www subdomain requests in AWS?

I have a question about how DNS works in conjunction with AWS s3 buckets that I am trying to understand. I have a modest knowledge about DNS and AWS, but I don't understand why my routing solution ...
Roger Hill's user avatar
0 votes
0 answers
319 views

Dropzone: How to upload video to S3 using presigned URL and play it in browser? Doesn't quite work

I have Dropzone set up to allow users to upload videos directly to an S3 presigned URL using the PUT method. Here is my Dropzone script: var presign = "#"; Dropzone.autoDiscover = false; var ...
peppy's user avatar
  • 73
0 votes
0 answers
368 views

minio A header you provided implies functionality that is not implemented

I'm trying to move object on MinIO yet running into weird issue (even though I'm able to move other objects fine without any issues) summary: mc: Failed to copy https://FQDN/path1/file.txt. A header ...
alexus's user avatar
  • 13.4k
0 votes
1 answer
146 views

Proper Setup for a simple AWS S3 website using CloudFront & Route, with base domain and www subdomain

Which is the better design, and why? (see 2 tables) 1. Here is the configuration of my working website (example.com is the prevent domain) as setup via a Youtube Tutorial... Route53 Cloudfront Dist ...
TheArchitecta's user avatar
0 votes
0 answers
114 views

Multi-region internal, domain resolved static website hosted in S3

Statement I do have a website that has a single page application hosted in S3, and a web service hosted in AWS Lambda. To avoid cross-site requests, both share the same domain, and I route the ...
Luís Guilherme's user avatar
0 votes
1 answer
471 views

How to set correct Content-Type HTTP header for all files uploaded on S3 bucket?

What is please the correct way to set Content-Type HTTP header for files uploaded on S3 bucket? We use S3 bucket as an origin server for hosting static websites. Bucket is accessed through the ...
STeN's user avatar
  • 1
0 votes
0 answers
128 views

Enable MFA S3 delete

I have THALES TOTP hardware tokens (MFA) which I would like to use as and additional protection against the accidental S3 object deletion (https://docs.aws.amazon.com/AmazonS3/latest/userguide/...
STeN's user avatar
  • 1
0 votes
0 answers
89 views

AWS AppStream is unable to push session script logs to s3

I have configured session scripts to log stdout and stderr to s3 bucket as described in https://docs.aws.amazon.com/appstream2/latest/developerguide/use-session-scripts.html#enable-S3-bucket-storage-...
MPP's user avatar
  • 1
0 votes
0 answers
142 views

AWS Lambda's indirect S3 requests

I noticed AWS Lambda causes S3 tier 1 and 2 requests. Do these cost anything after 12 months free tier expires? Specifically: Create new AWS Lambda function in AWS console causes 1 tier 1 S3 request ...
fejyesynb's user avatar
  • 121
0 votes
0 answers
386 views

botocore.exceptions.ClientError: An error occurred (InvalidToken) when calling the ListObjectsV2 operation

I have an mlflow server that is hosted on aws, that uses s3 in the background for storage. I am trying to run a lambda-function that runs a python script which accesses the mlflow server, loads a ...
ptushev's user avatar
0 votes
2 answers
299 views

How I can Log what and who can access and where my S3 bucket?

Recently I received an email in order to tell me that some IE connections has access to my bucket: eu-central-1|media.myapp | REST.GET.OBJECT|TLSv1|9|[Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1;...
Dimitrios Desyllas's user avatar