Questions tagged [amazon-s3]
This is for questions on Amazon's cloud storage service.
1
vote
3answers
42 views
How can you use Amazon S3 or S3-compatible Storage as NFS?
How can you use Amazon S3 or S3-compatible Storage as NFS?
Is there a way to achieve this?
Something that can be accessed through an external network (internet, public) using NFS protocol.
1
vote
2answers
28 views
Reverting S3 BucketPolicy
I was playing around with AWS S3 BucketPolicy. My intention was to allow access to bucket only to my username. I wrote following incorrect policy:
{
"Id": "Policy1542608345364",
"Version": "2012-...
0
votes
0answers
17 views
Cyberduck CLI: uploading public file to DigitalOcean Space/Amazon S3
I am trying to upload a file to DigitalOcean space with Cyberduck CLI https://duck.sh/ from Windows 10 box like that:
duck --upload digitalocean-ams3://myspace/dir/ test.txt
This works, but the ...
3
votes
2answers
34 views
Restrict access to S3 bucket folders to specific website users? (not using IAM Users)
I have a website where users need to log in. They can upload and delete their own pictures BUT these pictures are supposed to be private so images are not set to public that anyone can view.
I know ...
0
votes
0answers
15 views
Why does NGINX rewrite my url when I don't have a trailing slash? (url should not be rewritten)
I am trying to get a route such as example.com/misc/page to go to the same place as example.com/misc/page/. The route /misc goes to an s3 bucket that contains a bunch of other static pages that I am ...
3
votes
2answers
133 views
How to create an OFFLINE Incremental backup of an AWS S3 bucket
I'm looking for a way to take daily backups of an AWS bucket as incremental backups. These are to be stored offline and away from AWS.
For other storage systems (such as NAS drives) I use a daily ...
1
vote
1answer
25 views
Creating an S3 Bucket with Only Public Read and Download Permissions
I am trying to setup an S3 Bucket that has public read and download permissions, but not public upload.
I have the following permissions in the "Access Control List"
And Bucket Policy which allows ...
35
votes
2answers
7k views
AWS S3 charging for 4 TB of storage when only using less than 1 GB
I'm having trouble understanding my large S3 bill, and figured I'd ask here before dropping $30 on AWS monthly support.
Basically, I have an Amazon EC2 instance that makes an API to different ...
0
votes
1answer
33 views
InternalError when deploying to ECS using CodePipeline
I'm trying to deploy to ECS using AWS CodePipeline. Source and build stages are working fine, but deploy stage shows an error:
JobFailed: Internal Error
If I deploy my project to my ECS cluster ...
0
votes
1answer
30 views
Why does Amazon recommends reading 'all data' as soon as possible during S3 download?
While reading up official documentation on Amazon S3 Java SDK, I found an interesting note:
Your network connection remains open until you read all of the data or close the input stream. We ...
0
votes
0answers
12 views
Cloudfront with s3 file updates
I have an EC2 server with a website that allows logged-in users to upload very large files via a form for later download (these files are placed in a non-public s3 bucket). I want to use Cloudfront ...
1
vote
0answers
52 views
Setting up IIS reverse proxy to preserve host headers
I have an IIS server that is hosting a number of sites and apis. These sites include Confluence and Jira instances. These products actually run their own web servers so the Application Request Routing ...
2
votes
1answer
46 views
How to check which IAM performed an object modification of an s3 bucket
Is there a way to see which IAM did a modification on a aws s3 bucket object (with versioning enabled)?
I know that there is the aws s3api list-object-versions, but it only shows the owner's ...
2
votes
1answer
66 views
Backing up Apache access logs to S3
I am trying to backup Apache 2.4.27 logs on Amazon Linux 2016.03. I am trying to do so without writing a shell script, and I do not want to configure AWS access key and secret key on my Linux machine.
1
vote
3answers
41 views
Using S3 as a alternative to EFS for backup purpose
For centralised backup purpose of specific data from my instances, i was using EBS Volume mounted to that instances. Now we are planning to migrate to another region, but EFS service is not available ...
1
vote
2answers
26 views
AWS PowerShell Significantly Slower the AWS CLI
I'm having an issue in that a PowerShell Script takes 10 times as long as a batch file to download files from AWS S3.
I have an existing batch file script to move files from one S3 Bucket to another,...
0
votes
1answer
21 views
aws s3 cp files from the root of a subdirectory without getting subfolders as well? Powershell
s3://bucketname/yyyymmdd/foo/bar I have some files in the root directory and I want to copy them. I do not want to copy any subdirectories to the root (foo). I'm trying this in powershell:
aws s3 cp "...
0
votes
0answers
85 views
How to handle trailing slash in a redirect rule for an AWS S3 website?
I'm trying to build up some landing pages in my s3 hosted web site.
For example: http://www.example.com/products
That should redirect to http://www.example.com/products.html
To accomplish this, I ...
1
vote
1answer
23 views
Restricting S3 access to individual servers when using S3 endpoint in private VPC with no public IPs
Question
How can you restrict access to an S3 bucket to a single AWS instance when the instances have no public IP address and you're using an S3 endpoint in your VPC?
Background and Detail
We have ...
0
votes
0answers
27 views
Opsworks Stack and S3 access, can't get files.
For the life of me I can't figure out why I can't get opsworks to access the files in a bucket!
I have a chef 12 stack in opsworks. The repository is a zip file in an S3 bucket called "cheftest" (...
1
vote
1answer
63 views
Private s3 access from ec2 user-data script
I'm attempting to write an ec2 user-data script that will pull a file down from a private s3 bucket. The ec2 instances are located in multiple regions, which I believe eliminates the possibility of ...
1
vote
2answers
49 views
Determine which AWS account owns an S3 bucket
If someone gives me an S3 url with a bucket and an object, but I don't know which account owns the bucket, how can I determine which credentials to present in order to gain access to the object, ...
1
vote
1answer
127 views
Editable URL Rewrite Map for CloudFront/S3 Website using Lambda@Edge
I'm migrating a static website from IIS to Amazon S3, using CloudFront for https. The old site used the URL Rewrite module and had 301 redirect maps with over 200 entries. Here is an example:
example....
0
votes
0answers
35 views
log cloud foundry syslog to s3 using the syslog protocol
I want to send my syslog entries to an S3 bucket. Security restrictions have meant that this is the only ways that I can do this.
With cloud foundry, you can create a user provided server where I ...
0
votes
1answer
110 views
Amazon EKS: how to configure S3 access for worker nodes?
How can I configure an EKS cluster to automatically allow S3 access from worker nodes?
I've set up an EKS cluster following the Getting Started guide and have run the example Guest Book app. Now I ...
0
votes
2answers
53 views
Aws S3 encryption. Why use it?
I am not into the field at all. I am trying to understand what is the actual use of S3 encryption if the file is decrypted automatically when requested access to it.
Anyone with permissions to access ...
0
votes
1answer
43 views
Secure way to handle S3 uploads
I need to create an S3 bucket to host the assets uploaded by my users.
I've tried to create an IAM user dedicated to S3 which basically means:
- access only S3 and nothing else
- programmatically
- ...
2
votes
1answer
30 views
How to map subdomain in CloudFront to the same name in S3?
I have been searching for a way to do the following if someone can enlighten me would be greatly appreciated.
e.g. would the following mappings be possible in a single CloudFront instance?
feature-a....
0
votes
2answers
40 views
Making sure internal network is used
I am getting used to cloud computing, especially Amazon Web Services (AWS).
AWS has the concept of regions and availability zones. All availability zones of a Region are connected by a special, ...
1
vote
0answers
27 views
How to get s3 bucket policy update in cloud trail logs
Hi I have created a bucket and enabled cloud trail also including s3 object level api logging.
Now when I create some s3 ACL via UI the I get the logs in cloud trail but when I Create the same ACL ...
0
votes
0answers
212 views
How to use rclone to sync with s3 and acl public-read
Ref: Rsync to AWS S3 bucket
A reply to the above question suggests using rclone in a similar way to rsync. It seems like it might work, but it doesn't seem to use the ACL I want.
Previously, I use ...
0
votes
0answers
21 views
Backup AWS VOLUME to S3 (not EBS Snapshot on s3)
Backup AWS #Volume (to #S3 or similar).
Problem:
I have some old instances, and even worse, old data volumes, that needs to be stored for some time, then removed. Very likely noone would look at it ...
1
vote
1answer
190 views
`Access Denied` for some files, when syncing buckets
I'm using awscli to sync 2 buckets (same account):
aws s3 sync --only-show-errors s3://bucket-1 s3://bucket-2
but for some files I get permission errors:
copy failed: s3://bucket-1/dirname/file....
0
votes
0answers
24 views
OpenSSL X509 PEM Lib error Proxying to AWS S3
I am using Vormetric Cloud Encryption Gateway to proxy to AWS S3.
env | grep proxy
http_proxy=http://10.91.3.37:3128
https_proxy=https://10.91.3.37:3128
Issuing the aws s3 ls to return a listing of ...
0
votes
0answers
30 views
duplicity from macOs to S3 gives gaierror: [Errno 8] nodename nor servname provided, or not known
I'm backing up a macbook pro to AWS S3. It successfully writes the first volume, and then dies with
Attempt 1 failed. gaierror: [Errno 8] nodename nor servname provided,
or not known
I'm ...
0
votes
0answers
31 views
CloudFront does not handle “Origin Path” and origin redirections well
With Cloudfront we can define an "Origin Path" different from "/". It is useful when having multiple folders on a single S3 bucket for example.
Suppose you have many folders on a S3 bucket, websiteA, ...
4
votes
1answer
228 views
What does new “S3 Increased Request Rate Performance” annoucement mean
On 17 July 2018 there was an official AWS announcement explaining that there is no longer any need to randomize the first characters of every S3 object key to achieve maximum performance: https://aws....
0
votes
0answers
38 views
AWS: SES Rule: S3 Action: fails when choosing bucket that has encryption enabled
I can not create a SES rule to put emails into a S3 bucket that has encryption enabled (on the bucket).
Created a bucket and enabled encryption.
Add a SESPut bucket policy to allow SES. https://docs....
0
votes
1answer
36 views
AWS SES Encryption vs S3 bucket encryption
SES
Configuration for a SES rule allows one to have the email be put into a S3 bucket.
Under this configuration, the option "KMS Key" is available, which would have SES encrypt the email before ...
0
votes
0answers
49 views
architecture binary-amd64 not recognized in ubuntu mirror
I am using preseed to install ubuntu 16.04 and I have a setup that works fine with the regular ubuntu mirror if I specify
d-i mirror/http/hostname string archive.ubuntu.com
d-i mirror/http/directory ...
1
vote
0answers
248 views
Error importing VM to AWS : S3 key does not exist
I need to import a Gen2 Hyper-V machine to AWS and I'm following the document here: https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html
All works fine until I use this ...
0
votes
1answer
67 views
Amazon S3 to Amazon Glacier custom vault
So I was researching about Amazon Cloud services and how to backup all my personal infrastructure to the cloud -btw it's really amazing what Amazon has achieved on this field-.
I started looking ...
0
votes
0answers
22 views
How does creating an S3 endpoint affect other resources using S3?
I've just created a Lambda in AWS that requires access to RDS and, as a result, has to be a VPC Lambda function instead of a regular one. As I understand it, this means I basically have no choice but ...
0
votes
1answer
121 views
How do I limit S3 object access to CloudFront only?
TL;DR - How do I block requests for https://s3-eu-west-1.amazonaws.com/BUCKET-NAME/FILE.EXT while allowing requests coming via CloudFront?
I followed the AWS documentation "Restricting Access to ...
0
votes
0answers
90 views
Amazon S3 - Cloudfront - Images just randomly get permission denied
we have a very strange issue with the images on our websites. From time to time, some images get a permission denied response when our website is requested. These images are in a S3 bucket with ...
0
votes
1answer
98 views
Files Not Appearing in S3 File Storage Gateway Bucket
I created an AWS S3 File Storage Gateway and mounted it to a Linux server on-prem. On that same server where the file gateway is mounted I have a separate NFS share mounted. My goal is to copy files ...
0
votes
1answer
124 views
Serve a static website on S3 via CloudFront and ACM SSL without Route53
There is a great write-up of what I consider a "standard" setup of a single-page app hosted on AWS: S3 Static Sites. It includes
Deployment to S3
SSL certificate via ACM
Distribution via CloudFront
...
1
vote
0answers
32 views
Restore AWS Storage Gateway Server
I just setup an AWS File Storage Gateway server on-prem. I was able to connect it to an S3 bucket and I can mount it as an NFS share to Linux servers which is great. My concern though is this - if for ...
2
votes
1answer
140 views
How to use SAMBA on an Amazon S3 mounted directory on Ubuntu VM
I had successfully configured a SAMBA share on /home/myusername on an Ubuntu 17.10 VM. I then used s3fs-fuse to mount an S3 bucket on that exact directory, i.e. /home/myusername. It worked too, any ...
1
vote
1answer
37 views
S3 Bucket for hosting media, s3fs mount catch-22
I'm attempting to use an S3 bucket to host media files (images, video and static files) for a small web application and came across a strange catch-22...
According to the S3 documentation, to serve a ...
