Questions tagged [amazon-s3]

This is for questions on Amazon's cloud storage service.

286 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8 votes
1 answer
1k views

Amazon S3 appears to be blocking cloudflare IP addresses. How do I fix it?

I have static content that is being served by Cloudflare. Cloudflare points to Amazon S3 to pick up the static content and serves it via a CNAME (cdn.mydomain.com). The bucket is setup properly and ...
JStark's user avatar
  • 181
6 votes
0 answers
1k views

How does S3FS (or any other S3 FUSE filsystem) compare to AWS FSx for Lustre + S3

I remember trying s3fs a year back, trying to use some S3 bucket as a FUSE filesystem. I remember it being quite laggy, especially when coupled with git operations on it (an oblivious system architect ...
dimisjim's user avatar
  • 245
6 votes
1 answer
2k views

S3/Cloudfront CORS working on some files only

I'm really struggling to get Cloudfront and S3 to add Access-Control-Allow-Origin: * to the headers of video files stored on S3 (for inline video on iPhones - seems to be working everywhere else ...
beek's user avatar
  • 225
5 votes
0 answers
5k views

How to handle trailing slash in a redirect rule for an AWS S3 website?

I'm trying to build up some landing pages in my s3 hosted web site. For example: http://www.example.com/products That should redirect to http://www.example.com/products.html To accomplish this, I ...
101010's user avatar
  • 365
5 votes
1 answer
921 views

Syncing / backup from S3 buckets to openstack swift containers

I need to backup/sync few hundred GB of files in S3 buckets to openstack swift containers. I have thousands of files in several S3 buckets I want to copy/sync over openstack swift containers but I ...
ict4ngo's user avatar
  • 51
4 votes
0 answers
2k views

Understand S3 cost at folder level

I am planning for a use case wherein which my S3 bucket is used by 10 different users. All these users has separate folders within this bucket, to where they'll store their files. Now I want to know ...
serverstackqns's user avatar
4 votes
0 answers
3k views

How to remove orphaned s3 bucket

I have an s3 bucket that I can see in the console or with aws s3 ls/aws s3api list-buckets but the bucket appears to be in some sort of orphaned state. It's missing the Access and Region fields in ...
Robert's user avatar
  • 141
4 votes
1 answer
3k views

Does CloudFront support S3 signature version 4 for KMS encrypted objects?

I'm using Cloudfront with an S3 origin that is using KMS to encrypt objects. I'm getting the following error when sending a GET request for an object in the S3 bucket. Requests specifying Server ...
DJ Tarazona's user avatar
4 votes
0 answers
196 views

S3 + Cloudflare vs S3 + 2 Web Server + Cloudflate

Currently, I run two websites and each site own it own big server. Currently, all of those two sites has roughly 300GB of images. Its grow rate is 15GB of images per month. I know that eventually, one ...
Monkey D Luffy's user avatar
3 votes
0 answers
356 views

Downloading fails at random for our users from S3, need help figuring out what's wrong

We have a file sharing service where users can upload, share and download their files. We are using AWS S3 to store our files. We're facing an issue where small files download fine, but when files ...
Jahanzeb's user avatar
  • 131
3 votes
2 answers
274 views

AWS Opsworks Chef deployment using S3 Archive without making archive public

I'm trying to use the S3 Archive method to do a Chef deployment in Opsworks. I put the Chef archive in S3 and copied the address of https://s3.us-east-2.amazonaws.com/redacted/redacted.tar.gz. I gave ...
five_dollar_shake's user avatar
3 votes
1 answer
1k views

AWS: SES Rule: S3 Action: fails when choosing bucket that has encryption enabled

I can not create a SES rule to put emails into a S3 bucket that has encryption enabled (on the bucket). Created a bucket and enabled encryption. Add a SESPut bucket policy to allow SES. https://docs....
eugenevd's user avatar
  • 419
3 votes
0 answers
645 views

How to have frontend, on s3, and backend on another server with the same domain

I have the frontend part of an application on Amazon S3 and the backend on cloud66, with NGinx. The DNS have 'www.website.com' pointing on S3 and 'api.website.com' pointing on nginx. I want to use the ...
Dougui's user avatar
  • 167
3 votes
0 answers
2k views

CloudFront + S3 Access Denied Using Signed Cookies

I'm trying to integrate signed cookies for a CloudFront distribution that's fed off of an S3 bucket but getting access denied error messages: <Error> <Code>AccessDenied</Code> ...
YLivay's user avatar
  • 31
3 votes
0 answers
779 views

Set up a different index document per directory with S3 static website hosting

I am using S3 static website hosting, and trying to get the following redirects to work: example.com --> example.com/index.html example.com/foo --> example.com/foo/foo.html example.com/bar --&...
charles's user avatar
  • 147
3 votes
0 answers
524 views

Solution for file data backup, versioning and replication

We have a growing set of data files (.wav files, image files etc) which are data, i.e not part of the application code - uploaded and modified by users. The number of files is in the 1000s and the ...
Amir Abiri's user avatar
2 votes
1 answer
486 views

Since S3 charges by request, couldn't a malicious hacker cause a huge AWS bill just by spamming requests?

What would stop them from doing so, against, say, a static website hosted using S3? Is there a good way to deny some requests such that one avoids getting billed for them? (Context: I want to host a ...
Asker's user avatar
  • 121
2 votes
0 answers
1k views

CloudFront CORS request using signed cookies and withCredentials, not sending back Access-Control-Allow-Credentials unless I include some extra header

I'm having a very strange issue that I can't seem to crack. I configured a private CloudFront distribution to serve content from a private S3 bucket. I am using signed cookies to grant access to the ...
bryan60's user avatar
  • 71
2 votes
0 answers
2k views

Access denied for S3 bucket with redirect access via CloudFront

I have public S3 bucket exposed via CloudFront to S3 A record for www.mezinamiridici.cz. https://www.mezinamiridici.cz/ - works http://www.mezinamiridici.cz/ - redirects to https://www.mezinamiridici....
Leos Literak's user avatar
2 votes
0 answers
394 views

gsutil rsync fails on Ubuntu with UnicodeDecodeError

I'm trying to rsync between google cloud storage and amazon s3. On my local machine (MacOS) this command works gsutil rsync -d -r gs://mira-internal/data/exports/20191108 s3://mira-temp/raw/20191108 ...
kellanburket's user avatar
2 votes
0 answers
446 views

S3 GetBucketAccelerateConfiguration operation: Access Denied

I need to grant user access to get AccelerateConfiguration for one bucket. I set next permissions: { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "...
ZVErius's user avatar
  • 21
2 votes
1 answer
620 views

Use S3FS Mount As HTTP Server Document Root

My proposed setup: An EC2 server running Apache or nginx. An S3 Bucket in the same region/availability zone mounted with S3FS. The Apache or nginx document root set to the S3FS mount. My question: ...
xendi's user avatar
  • 414
2 votes
0 answers
699 views

Server access logging on AWS S3

Delving into the AWS world, I have created two buckets in my AWS. The first one called BucketA (holding the files) and the second one called BucketB (holding the log files). BucketA will be hosting ...
tony9099's user avatar
  • 167
2 votes
1 answer
2k views

Cloudfront (w/ S3) Static Website Doesn't utilize index.html from sub directories

I have my website hosted on S3 with CloudFront as a CDN, and I need these two URLs to behave the same and to serve the index.html file within the directory: example.com/directory example.com/...
Costa Michailidis's user avatar
2 votes
1 answer
114 views

Serving backup/fail-safe object on AWS S3

What would be the easiest way to serve backup/fail-safe object when the originally requested object is not available in the bucket? For example, currently our users can upload custom profile photos ...
choxnox's user avatar
  • 21
2 votes
0 answers
847 views

log cloud foundry syslog to s3 using the syslog protocol

I want to send my syslog entries to an S3 bucket. Security restrictions have meant that this is the only ways that I can do this. With cloud foundry, you can create a user provided server where I ...
dagda1's user avatar
  • 247
2 votes
1 answer
869 views

AWS SES Encryption vs S3 bucket encryption

SES Configuration for a SES rule allows one to have the email be put into a S3 bucket. Under this configuration, the option "KMS Key" is available, which would have SES encrypt the email before ...
eugenevd's user avatar
  • 419
2 votes
0 answers
21 views

Proper way to handle file serving

We have a website where we our daily bandwidth use is roughly 10GB of user generated contents. The software layer is simply Heroku + some addons for DB and monitoring, and we are handling the uploads ...
nakajun's user avatar
  • 21
2 votes
0 answers
915 views

How to get clean URLs in CloudFront

I'm hosting my static website on AWS S3, with Cloudfront as a CDN, and I'm wondering how I can get clean URLs working. I currently have to go to example.com/about.html to get the about page. I'd ...
Costa Michailidis's user avatar
2 votes
0 answers
746 views

rpc.mountd[pid]: /xxx and /yyy have same filehandle for host, using first

I have an S3 bucket mounted using s3fs-fuse at /yyy and a normal drive mounted at /xxx. I am exporting both with NFS however when a client attempts mount, I see the following error: rpc.mountd[pid]: /...
Alex Rothberg's user avatar
2 votes
1 answer
1k views

Can I control frequency of AWS S3 bucket access log delivery?

I have enabled access logging for an AWS S3 bucket. Now I get log files delivered to my chosen destination. Each file has a name of the form YYYY-MM-DD-hh-mm-ss-guidguidguid with the time representing ...
Sparr's user avatar
  • 809
2 votes
0 answers
780 views

Reverse Proxy on cloudfront

I have an angular site / web application (Lets call this www.mysite.com) hosted on Amazon S3 as a static website. This s3 bucket serves as the origin for a cloudfront distribution and it works ...
Anomaly211's user avatar
2 votes
1 answer
697 views

Route 53 configuration for Cloudfront distribution linked to S3 bucket for naked domain

I am using route 53 for a domain (example.com) linked to my app on Heroku. To handle naked domain redirection, I am using a S3 bucket as explained here: https://aws.amazon.com/blogs/aws/root-domain-...
Cyril Gaillard's user avatar
2 votes
1 answer
2k views

How backup files with logrotate and Amazon S3?

I would like to create a new standard of two things: how long would be generated and be rotated the logs generated by my applications ? how to transfer the logs to Amazon S3, as a backup server ?. ...
Valter Silva's user avatar
2 votes
1 answer
1k views

Amazon S3 Website & MySQL Backup

I have a website (a digital asset management system/gallery - http://www.resourcespace.org) that has a huge amount of images. The total size of the website, including the images is approximately 6gb. ...
user avatar
2 votes
0 answers
634 views

s3fs-mounted S3 bucket shows bad filesizes

Background: I have an Amazon S3 bucket with data uploaded using the AWS web interface. Now I need to give permissions-based access to paths within my buckets to FTP users. To do this, I'm running ...
jonaz's user avatar
  • 131
2 votes
0 answers
323 views

Gluster with s3ql mounted brick?

I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount. Having just the plain directory as a unique ...
mr-euro's user avatar
  • 858
2 votes
0 answers
941 views

Web-based file manager like File Thingie, but for S3?

At work, we use a little PHP script called File Thingie, which is a web-based file manager. Basically, an FTP replacement. It allows users to upload and download files, and most importantly, it has a ...
Legion's user avatar
  • 233
2 votes
1 answer
740 views

How to access an Eucalyptus Walrus(S3) using Amazon AWS SDK

I have downloaded the Amazon AWS SDK for C#, I have no problem accessing the EC2 part of our private cloud running Eucalyptus, I can list, Images, Instances, Zones ... This is working fine : ...
pallaire's user avatar
  • 110
1 vote
0 answers
427 views

Amazon S3 life cycle rule for sub folder contents?

With this reference link My s3 bucket name say testbucket. If I create prefix for a rule say. subfolder1/ All the objects in it are applied this rule. But it is not getting applied for objects inside ...
user2331760's user avatar
1 vote
0 answers
357 views

AWS S3 Bucket Static Website Index/ Home Access Denied

So i have problem with AWS S3 Bucket Static when i try to access my domain without filename. http://customdomain/ >> Error access denied http://customdomain/index.html >> normal but it's ...
Sinauwae's user avatar
1 vote
0 answers
209 views

making all S3 uploads to digitaloceans spaces default to being public

I'm using DigitalOcean Spaces to upload a bunch of files in different directories. In particular, I'm using WinSCP to do the upload via the S3 protocol. The problem is that, per WinSCP, all the files ...
neubert's user avatar
  • 337
1 vote
0 answers
66 views

Reduce Query Syntax for ends_with()

I have the following query: --query "Contents[?ends_with(Key, 'jpg') || ends_with(Key, 'NEF') || ends_with(Key, 'nef') || ends_with(Key, 'CR2') || ends_with(Key, 'cr2')]" Is there a way to ...
davidbitton's user avatar
1 vote
0 answers
279 views

How suitable is a S3 backend for rsync operations?

I run a large server providing open source software (https://ftp.halifax.rwth-aachen.de), currently serving more than 30 TByte of data with Multi-Gigabit throughput. Data is synchronized and kept up-...
C-Otto's user avatar
  • 374
1 vote
0 answers
162 views

Digital Ocean Space Cors with private files

I am unsure whether I am misunderstanding CORS or whether this is possible within a Digital Ocean Space. Basically, I want to set a file to private on the DO Space. I only want a certain domain to be ...
Soup Dragon's user avatar
1 vote
0 answers
922 views

Unable to resolve AWS S3 and other DNS from Docker container

We have an API that puts an object to AWS S3 using the Java API. It works fine from the locations below: From our local dev workstations From within production docker containers in EC2 machines. But,...
Sriram Sridharan's user avatar
1 vote
1 answer
1k views

S3 access control based on bucket tags

i hope you can help me out. I have read a couple of docs now, and I am still unsure whether this actually works. I want to give access in different levels to AWS users based on S3 tags. Example: S3 ...
flypenguin's user avatar
1 vote
1 answer
115 views

S3 policy to allow by IP can't be changed

We have an S3 bucket with website content in it that's served to the public through a proxy cache. The access to the bucket is limited to the IPs used by the proxy. This is all working fine. New IPs ...
Tim Tisdall's user avatar
1 vote
0 answers
111 views

Calculation of S3’s byte-hour

S3 uses the concept of Byte-Hour in billing: https://aws.amazon.com/s3/faqs/#Billing Unfortunately (in typical Amazon AWS style) they left some details unsaid, namely what happens in the case of ...
zebradark's user avatar
1 vote
0 answers
154 views

How do I block TCP ACK causing retransmissions

I am trying to replicate a issue when making requests to S3 that will lead to tcp_retries2 leading to 900 seconds of retransmission attempts before giving up. How would I do this? Is it sufficient to ...
user3130197's user avatar

1
2 3 4 5 6