Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
286
questions with no upvoted or accepted answers
1
vote
0
answers
7k
views
Intermidiate "upstream prematurely closed connection while reading upstream"
I am getting an intermediate "upstream prematurely closed connection while reading upstream" errors on my server.
The error only occurs on the data that is requested throe the server from Amazon S3 ...
1
vote
1
answer
562
views
Allow access to S3 bucket via VPC or via console?
I want my S3 bucket to be accessible in two ways:
Via the console when I'm logged in, so I can modify files
Via EC2 instances in a certain VPC (IAM roles are set up correctly for this)
Here's the ...
1
vote
2
answers
510
views
AWS/S3 - creating a user (through console) to grant r/w bucket perms to for 1 bucket?
I have an Amazon Web Services S3 bucket I want to use with the android data syncing app 'FolderSync'.
Towards that I want to set limited perms on the bucket for a new user.
Within the AWS management ...
1
vote
1
answer
843
views
Amazon Cloudfront with S3 bucket - 2 origins
I am bit confused with setting up Amazon Cloudfront and S3 bucket.
I've got my bucket setup and using the wordpress WP Offload S3 plugin to store any images in this bucket. When checking the image ...
1
vote
0
answers
880
views
Amazon Elastic Beanstalk with File Access to Amazon S3
i've a problem with my Amazon Elastic Beanstalk and Amazon S3 setup.
From the EB instance i want to load a file with node (fs.readFileSync), i tried a ton of configurations - but non of them worked, ...
1
vote
1
answer
2k
views
s3cmd error on sync: S3 error: 400 (InvalidRequest)
I'm trying to configure s3cmd to backup some directories on my server to my AWS S3 bucket. My problem is that when I do the sync command: s3cmd sync -r -p --no-delete-removed /path/to/dir s3://...
1
vote
0
answers
55
views
fast backup script in linux
I have a server with more applications and do normal shell command to take backups. it took couple of days to complete. Tried to install duplicity. but its told some packages are not found. its Debian ...
1
vote
0
answers
33
views
Enabling S3 bucket as website using cli
I figured out how to create an S3 bucket using aws command line. Is it possible to enable the s3 bucket I created as website using aws cli? The reason I'm doing this is because we're automating ...
1
vote
0
answers
1k
views
Bucket policy that respects pre-signed URLs OR IP Address deny?
I would like to be able to restrict access to files in a S3 bucket in multiple ways. This is due to the fact that the files stored can be accessed in different manners. We do this because we have TBs ...
1
vote
0
answers
2k
views
Access denied for the request of data from s3 bucket on high load
I am trying to fetch static files from s3 bucket in case of 502 error.
This is how I have configured nginx for requests :
error_page 502 = @static;
location / {
index index.php;
try_files $uri $...
1
vote
0
answers
8k
views
AWS security group egress rules for S3
I manually created a new security group using the AWS CLI.
I created ingress rules that allow incoming connections only from my company's public IP address using the known ports for SSH (22) and ...
1
vote
1
answer
2k
views
Can I use an IAM role to grant my Heroku app access to my Amazon S3 bucket?
Heroku: Using AWS S3 to Store Static Assets and File Uploads suggests using my AWS security credentials to enable my Heroku app to access my Amazon S3 bucket.
However, isn't it better practice (as ...
1
vote
1
answer
429
views
AWS S3 bucket permissions
I've created AWS s3 buckets with several thousand objects, initially the access policy was restricted to authenticated users, but later due to some requirement we need to make all objects in this ...
1
vote
1
answer
1k
views
jboss cluster in ec2 with s3ping
Am trying to get s3ping discovery method working in wildfly 8.0.0. I am using default Amazon Linux AMI and using chef to launch instances in ec2 based on this tutorial http://aws.typepad.com/...
1
vote
2
answers
2k
views
AWS S3 Sync options duplicates / weird behaviour
I am trying to sync a folder with 2M+ files to S3, everything went OK but then 40,000 files were not uploaded (server crashed randomly), when I tried to do a sync command again, it started from 0, ...
1
vote
1
answer
972
views
Nginx proxy cache invalidating request to s3
I have set up a proxy from my application server to a private s3 bucket to cache requests. I was having some trouble with it where s3 was rejecting my download requests (403 forbidden) and after some ...
1
vote
0
answers
100
views
svnadmin dump returns error insufficient disk space
I have a SVN repo that is around 35GB after dumping. I know this is huge, but our business requires that all revisions of all files and libraries be tracked.
I have Apache Subversion hosted on an ...
1
vote
0
answers
455
views
Duplicity -> S3 backup failing with "[Errno 105] No buffer space available"
I have a nightly script on my Ubuntu 10.04 VPS which uses duplicity (0.6.24) to run incremental, encrypted backups to Amazon S3. This script had been working until a month or so ago when it started ...
1
vote
1
answer
684
views
Glacier retrieval notification of s3 objects
I have some s3 buckets which send their objects to glacier after x days of creation. When I go to retrieve those objects on the s3 interface, I need to keep reloading page to see when restoration has ...
1
vote
0
answers
2k
views
Amazon S3: allow user access to subfolder only on any bucket
I'm trying to write a policy that will allow a user to read from any bucket (that part works) but only write to a specific subfolder on any bucket.
This is my policy so far (reading works, but it ...
1
vote
0
answers
336
views
Handling multiple server blocks in nginx configuration
I am working on setting up a proxy server for my application.The use case is I want a specific set of requests to be redirect to say abc.com and rest of the request to be redirected to xyz.com.
Eg:If ...
1
vote
0
answers
579
views
Amazon S3 sync theory
I'm setting up a backup of my vital data (database and images, mainly) to S3 from my Ubuntu box. I've set up Amazon S3, and installed S3FS to mount the bucket on my machine. This is working well, but ...
1
vote
1
answer
1k
views
Can AWS S3 files in buckets get deleted by a download timing out?
A few AWS S3 files might have gotten deleted while trying to download everything for a local backup. The download/connection timed-out a couple times. The next day several HTML/JS files are missing ...
1
vote
1
answer
897
views
Reducing latency for different geographic regions on Amazon Cloud
I have got an application which has three components
Application code : Amazon EC2 US-EAST-1 instance
Application images, and other static data : Amazon S3 with CloudFront
Application Database : ...
1
vote
1
answer
803
views
Why would anyone disable S3 KMS "bucket key" option?
As per the documentation:
Amazon S3 Bucket Keys reduce the cost of Amazon S3 server-side
encryption using AWS Key Management Service (SSE-KMS). This new
bucket-level key for SSE can reduce AWS KMS ...
1
vote
1
answer
3k
views
Setting cache for already uploaded amazon s3 files
In amazon s3 for our site through s3cmd tool we uploaded around 60GB of files including mp3, jpg to a bucket. But now when we checked site slowness files are not caching and further checking we came ...
0
votes
1
answer
18
views
Archive files in EC2 Instance
I want to create the archives of the files in EC2 instance to load them into the S3 bucket, but I am not getting a proper command to run on the AWS cli.
0
votes
0
answers
47
views
Velero cannot access same S3 from different EKS cluster
I have velero installed on two different EKS clusters in the same region. All velero installation configurations are same and installed using same charts and credentials, and uses same S3 bucket with ...
0
votes
0
answers
72
views
How to Deploy React App and WordPress on the Same CloudFront Distribution Domain Name with Different Origins and Behaviors?
I'm encountering challenges deploying both a React app and a WordPress site on the same CloudFront Distribution domain name while utilizing different origins and behaviors.
Here's my setup:
I have a ...
0
votes
1
answer
56
views
How to setup a S3 backed Media server with basic auth / password protection
Requirement
Serve audios and videos stored in a S3 bucket
Selectively protect folders with basic auth (username password);
Low latency as much as possible but not necessary
Cost is also a factor but ...
0
votes
0
answers
49
views
How to get CloudTrail to record user-data script in the log
I have users who will be able to submit scripts to instances via the aws ec2 run-instances --user-data file://bla.sh option.
I have setup CloudTrail to save almost all of the data that I need to log ...
0
votes
2
answers
282
views
Configuring S3 bucket as alternative storage for Dovecot sdbox
I'm trying to use an S3 bucket for long-term storage for emails, served with Dovecot, with a cron job to doveadm altmove everything older than a few weeks to alternative storage. I've been able to ...
0
votes
1
answer
49
views
Choosing AWS hosting for flutter website
Choosing AWS hosting for flutter website
I have a flutter website and i have been thinking to host the site in
AWS.
I have saw the below features in aws which I can use for hosting
my website.
AWS ...
0
votes
0
answers
72
views
Delete object everyday in S3 bucket
i'm trying to delete folders(objects) in the S3 bucket everyday (with lifecycle rule). I have this scenario :
TestBucket -> connect/ -> test/ -> 2023/ -> 12/ -> 01/, 02/, 03/ etc.
I ...
0
votes
1
answer
395
views
My Cloudfront distribution is showing access denied despite enabling all public access to the bucket objects
I am using the following CF policy in my S3 bucket:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"...
0
votes
1
answer
200
views
S3 site in Cloudfront Preflight 302 Error
I have a static s3 website hosted through CloudFront. In it, I make a request that will result in a redirect if the user is not logged in. Preflight then fails because of a 302:
Access to fetch at '...
0
votes
1
answer
281
views
Best method to copy image files from AWS EC2 instance to S3 bucket based on query
What is the most efficient method to copy selected image files from a EC2 instance (Ubuntu 20.04) in production to a S3 bucket while checking if the file exists in EC2?
This is a one-time operation. ...
0
votes
0
answers
121
views
Oracle MySQL operator for kubernetes backup to S3 using service account
I've installed the operator, and trying to create the innodb cluster with backup profile which uses S3. There is no mention of S3 in the documentation, but they mentioned in a conference about S3 ...
0
votes
0
answers
94
views
Maximize duplicity efficiency
I'm creating incremental backups for a database. The size is around 8 GB per dump.
I have a backup creation policy, that looks like this: (don't worry, I already avoided collisions)
An increment ...
0
votes
0
answers
115
views
Why do I need to map separate s3 buckets for www subdomain requests in AWS?
I have a question about how DNS works in conjunction with AWS s3 buckets that I am trying to understand. I have a modest knowledge about DNS and AWS, but I don't understand why my routing solution ...
0
votes
0
answers
319
views
Dropzone: How to upload video to S3 using presigned URL and play it in browser? Doesn't quite work
I have Dropzone set up to allow users to upload videos directly to an S3 presigned URL using the PUT method. Here is my Dropzone script:
var presign = "#";
Dropzone.autoDiscover = false;
var ...
0
votes
0
answers
368
views
minio A header you provided implies functionality that is not implemented
I'm trying to move object on MinIO yet running into weird issue (even though I'm able to move other objects fine without any issues)
summary:
mc: Failed to copy https://FQDN/path1/file.txt. A header ...
0
votes
1
answer
146
views
Proper Setup for a simple AWS S3 website using CloudFront & Route, with base domain and www subdomain
Which is the better design, and why? (see 2 tables)
1. Here is the configuration of my working website (example.com is the prevent domain)
as setup via a Youtube Tutorial...
Route53
Cloudfront Dist
...
0
votes
0
answers
114
views
Multi-region internal, domain resolved static website hosted in S3
Statement
I do have a website that has a single page application hosted in S3, and a web service hosted in AWS Lambda. To avoid cross-site requests, both share the same domain, and I route the ...
0
votes
1
answer
471
views
How to set correct Content-Type HTTP header for all files uploaded on S3 bucket?
What is please the correct way to set Content-Type HTTP header for files uploaded on S3 bucket? We use S3 bucket as an origin server for hosting static websites. Bucket is accessed through the ...
0
votes
0
answers
128
views
Enable MFA S3 delete
I have THALES TOTP hardware tokens (MFA) which I would like to use as and additional protection against the accidental S3 object deletion (https://docs.aws.amazon.com/AmazonS3/latest/userguide/...
0
votes
0
answers
89
views
AWS AppStream is unable to push session script logs to s3
I have configured session scripts to log stdout and stderr to s3 bucket as described in https://docs.aws.amazon.com/appstream2/latest/developerguide/use-session-scripts.html#enable-S3-bucket-storage-...
0
votes
0
answers
142
views
AWS Lambda's indirect S3 requests
I noticed AWS Lambda causes S3 tier 1 and 2 requests. Do these cost anything after 12 months free tier expires?
Specifically:
Create new AWS Lambda function in AWS console causes 1 tier 1 S3 request
...
0
votes
0
answers
386
views
botocore.exceptions.ClientError: An error occurred (InvalidToken) when calling the ListObjectsV2 operation
I have an mlflow server that is hosted on aws, that uses s3 in the background for storage. I am trying to run a lambda-function that runs a python script which accesses the mlflow server, loads a ...
0
votes
2
answers
299
views
How I can Log what and who can access and where my S3 bucket?
Recently I received an email in order to tell me that some IE connections has access to my bucket:
eu-central-1|media.myapp | REST.GET.OBJECT|TLSv1|9|[Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1;...