The amazon-s3 tag has no wiki summary.
1
vote
1answer
36 views
Serve up s3 file with nginx
I have some static files sitting in an s3 bucket that I want to serve up from my nginx server. Basically, if a user goes to www.mywebsite.com/this/url I want to server them up a file from the S3 ...
-1
votes
0answers
23 views
Connect FreeNAS and AWS S3 Service [closed]
I'm looking for a way of having my FreeNAS backup it's files to an AWS S3 bucket, using AWS Storage Gateway. Any ideas?
Thanks!
0
votes
1answer
19 views
Amazon S3 - can't upload files into the same folder using s3sync.exe
I'm trying to upload a file using S3sync.exe into a certain folder in my S3 bucket.
This is the command line : S3Sync.exe -AWSAccessKeyId accKey -AWSSecretAccessKey secKey -RequestEndpoint ...
1
vote
1answer
25 views
How to list s3fs mounts?
How can I list the s3fs mounts that exist on an ubuntu system?
I'd like to know to which bucket each mount is mapped.
Specifically, I have a specific mount (e.g. ~/s3/mymount), and would like to know ...
2
votes
0answers
36 views
Gluster with s3ql mounted brick?
I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount.
Having just the plain directory as a unique ...
0
votes
1answer
52 views
Amazon S3 - Automated Backup Using S3Sync, Windows Schedule & BAT file
I am trying to build an automated background backup program to backup my entire website and its heavy photo folders, using Amazon's cloud storage service, S3.
I am using S3Sync from S3Tools and have ...
0
votes
4answers
91 views
Easiest to use backup software for live linux server [closed]
I need a backup software that is capable of:
doing a full HDD image (from live server, if that's possible at all?)or just copy everything except /dev, /tmp, etc
updates the backup incremental just ...
2
votes
2answers
116 views
Which is the fastest way to copy 400G of files from an ec2 elastic block store volume to s3?
I have to copy 400G of files from an elastic block store volume to an s3 bucket... Those are about 300k files of ~1Mb
I've tried s3cmd and s3fuse, both of them are really, really slow.. s3cmd ran for ...
-2
votes
0answers
56 views
I installed s3fs and mounted my s3 bucket on EC2. How do i test if its working correctly
I installed s3fs and mounted my s3 bucket on EC2. How do i test if its working correctly?
I don't get any error when i mount, but I cannot see the s3 bucket files. Any help will be appreciated. when ...
1
vote
0answers
14 views
php script ran from crontab times out if other scripts are ran at the same time [migrated]
I have a server in my room with a duo-core processor and basic internet (nothing fancy)
I'm running ubuntu 11.04 and I'm running a php script from the crontab.
0 3 * * * php back_up_to_s3.php
This ...
0
votes
0answers
41 views
apache/squid as caching proxy for S3 over ssl
We have setup up a apache reverse proxy in conjunction with squid as a caching proxy for amazon S3. A server application is communicating with this proxy layer and caching works just fine saving ...
-2
votes
1answer
96 views
How do I calculate my bandwidth requirements?
We have a facility that needs to upload approximately 20 3MB photos per minute to Amazon S3. These are coming from 10 different machines on our local network. We need to do this continuously for 16 ...
scalability
2
votes
2answers
146 views
How do I create a EC2 S3 backed instance, starting from a Windows VHD?
I have a Windows VHD image created in Hyper-V, and I want it to exist as a S3-backed instance on EC2. I've found several articles (and a tutorial) on how to import my virtual machine to EC2. However, ...
2
votes
1answer
61 views
How can I turn an existing Windows Server 2008 EBS-backed instance into a S3-backed (instance-backed) AMI?
I understand there is no direct conversion to get from EBS to S3. But is there some series of steps that can make this happen, or is rebuilding the image from scratch the best option?
Is there a way ...
1
vote
1answer
56 views
Duplicity not writing to a pre-existing S3 bucket
I'm trying to backup a directory to a pre-existing Amazon S3 bucket using the following command:
duplicity --no-encryption system/ s3+http://MY_BUCKET_NAME/backup
However, I'm getting the following ...
1
vote
2answers
53 views
Configure non-destructive Amazon S3 bucket policy
There's a bucket into which some users may write their data for backup purposes.
They use s3cmd to put new files into their bucket.
I'd like to enforce a non-destruction policy on these buckets - ...
-4
votes
1answer
50 views
Where to store social network photos and videos? [closed]
I have a social network with 100,000 users. I allow users to upload photos and videos now I feel like it is becoming too much for my dedicated server. Should I get another dedicated server? A content ...
1
vote
2answers
228 views
Why is uploading to S3 so slow?
I am using s3cmd to upload to S3:
# s3cmd put 1gb.bin s3://my-bucket/1gb.bin
1gb.bin -> s3://my-bucket/1gb.bin [1 of 1]
366706688 of 1073741824 34% in 371s 963.22 kB/s
I am uploading ...
1
vote
1answer
163 views
Deleting multiple objects in a AWS S3 bucket with s3curl.pl?
I have been trying to use the AWS "official" command line tool s3curl.pl to test out the recently announced multi-object delete. Here is what I have done:
First, I tested out the s3curl.pl with a ...
0
votes
0answers
49 views
Looking for a Place for static website like Amazon S3 [closed]
I like to host some of my small static website on S3
because (pros):
very cheap
no maintenance need
reasonably fast (even without CDN)
every website can have a different IP, even in same zone, most ...
0
votes
1answer
75 views
AWS instances: AMI bundles and snapshots and bears, oh my
I've googled a good amount and am still pretty confused and feeling like I'm missing or overlooking something. My goal is to restore a host using a bundle.
I have an AMI bundle of a now-nonexistent ...
0
votes
1answer
22 views
apache sitemap rewrite to s3
I wanted to submit sitemap to google webmaster,
google state only accept sitemap from my site
Option 1 - simple rewrite
I know this already working:
RewriteRule /sitemap.txt http://S3_URL [L]
...
1
vote
1answer
59 views
Amazon CloudFrount on EC2
I know that i can mount s3 buckets with s3fs on EC2 instance. So i have next question can i somehow mount Amazon Cloud Front (my S3 buckets files) in EC2 instance?
1
vote
2answers
134 views
Snapshots == EBS backed instance, persistence, and S3 storage?
I want to clear up some confusion and clarify understanding on EC2 instances being EBS backed.
If you start an AMI from a community image, configure your server as you want it configured, then use ...
1
vote
3answers
90 views
CloudFront - Users requests static files non-stop to make your bill high?
I had this weird wondering once.
If I am using CloudFront as my CDN, I pay per requests made on my static files.
So here comes the wondering. Can one keep requesting the same file in a non-stop ...
1
vote
2answers
148 views
How to calculate Amazon S3 prices
I am trying to estimate how much I need to pay to store images for my customers on Amazon S3.
I use their calculator but the total seems to cheap. Feels like I am missing something.
Each image is ...
2
votes
2answers
284 views
What design features make Joyent's ZFS and Amazon's EBS (S3) reliable?
I know this isn't exactly an apples to apples comparison, what I'm trying to evaluate is which one is safer - less likely to lose data.
Joyent's SmartOS uses ZFS to store data whereas an EC2 machine ...
1
vote
0answers
165 views
Use Microsoft DPM 2010 with Azure or Amazon S3?
Does anyone know of any software that hooks up with Microsoft's DPM 2010 or even the new 2012 to ship data to cloud storage providers like Azure or S3?
We utilize D2D backups today but want a more ...
1
vote
2answers
68 views
How to show a “linked from site X” placeholder instead of actual photo?
I’m hosting a bunch of photos on Amazon S3 and also running Amazon CloudFront for them. Also, I’m running a custom domain name.
Now, in case someone tries to embed the image to another website, I’d ...
0
votes
1answer
111 views
Using an IP Address to Point to Amazon S3 Storage
Is there a way to get an IP address to refer to Amazon S3 Storage location? The current solution my company is thinking of going with involves setting up a EC2 server with an Elastic IP and using ...
2
votes
1answer
65 views
S3 to cloudfront edge location vs cloudfront to end user transfer ratio
We host videos in amazon S3 and use cloudfront to deliver them. Here is last months numbers:
4,2 TB total traffic from cloudfront to end user. 4 TB of this is in europe, the rest mainly in the US.
...
-1
votes
2answers
71 views
Cross platform, blob storage system
I have a couple of different servers around the house, and a lot of storage on each of these machines. I also have a couple of dedicated servers, and some VMs on these boxes... Most of the boxes and ...
0
votes
1answer
56 views
How do I adjust this python script to backup to Amazon S3
The following script is nearly perfect for my backup needs: http://pastie.org/3368661
However, it only allows you to backup to the server or a remote FTP. I would like to add an option to backup to ...
1
vote
2answers
202 views
Any need to make backup of data on Amazon S3?
I'm hosting 200 GB of product images at S3 (this is my primary file host).
Do I need to back that data up somewhere else, or is S3 safe as it is?
I have been experimenting with mounting the S3 ...
0
votes
1answer
90 views
A uploaded file is 0 byte on Amazon S3 using django
I've symbolic linked a S3 directory on my EC2 server. But a uploaded file was 0 byte when I uploaded file. It was work well before (I used just web server directory)
I don't have any idea dealing ...
0
votes
0answers
27 views
Changes applied to certain php files not reflecting at runtime
Problem
Today, all of a sudden, we found that certain things in our LAMP based web application are not displaying results. So, just to debug the issue, we modified some files and found that applying ...
-2
votes
1answer
47 views
Monitoring Amazon s3 data transfer
I'm hosting a few files on Amazon s3. These are just images for my website. Lately the traffic to my website has been growing. I'd like to monitor my data transfer from s3. How can I do this?
I've ...
3
votes
2answers
83 views
Tracking down “Connection reset” errors in Linux
I'm handling large numbers of concurrent downloads (approx. 500 per server) using Java.
All the files are being downloaded from Amazon S3, and the downloading server is an EC2 m1.large instance.
...
2
votes
2answers
790 views
Is mounting S3 buckets directly into EC2 instances safe?
as the title of my question says, I would like to know whether this is a good practice or not,
I know I can mount it in an EC2 instance but I was wondering if, as I will be adding more instances to ...
1
vote
3answers
154 views
projects or packages to emulate a tape drive with a Amazon S3 or OpenStack Swift compatible backend?
I've got some data that is being backed up to tape via bacula, and at some point soon I am going to need to migrate to non-tape storage.
Bacula supports to-disk archiving, but it occurred to me that ...
0
votes
1answer
88 views
Does anyone have a config to process AWS CloudFront/S3 logs with AWStats (or similar sw)?
I don't want to give an external company like s3stat access to my logs. I know that AWS logs S3 and Cloud Front in a format readable by AWStats. Has anyone used AWSats to analyze them?
S3stat used ...
-1
votes
1answer
72 views
Which of snapshots I can safely remove?
If you make periodic snapshots of a volume, the snapshots are
incremental so that only the blocks on the device that have changed
since your last snapshot are incrementally saved in the new ...
0
votes
0answers
87 views
Moving to the cloud for product building and testing of large Java product that uses databases [closed]
I currently build and test a large Java product based on Eclipse an Ubuntu machine using Jenkins (Hudson) and tested on both Ubuntu and Windows, the testing also involves all of the major databases. I ...
1
vote
2answers
664 views
How to move files between two S3 buckets with minimum cost?
I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone.
How could I ...
0
votes
1answer
78 views
Is uploading a 500mb file to s3 an expensive operation? Any way to limit cpu usage etc?
On a production server, I'm planning on uploading a single file or a set of files that may be 500 megs in size.
I havent' test it just yet, but just want some opinions on whether you think this will ...
0
votes
0answers
58 views
PostgreSQL and S3QL for storing/accessing lots of data
We're currently using Postgres 9 on Amazon's EC2 and are very satisfied with the performance. Now we're looking at adding ~2TB of data to Postgres, which is larger than our EC2 small instance can ...
2
votes
2answers
405 views
Forcing CloudFront to pass-through the latest HTML file from S3
Background
I'm hosting a static site on S3, with CloudFront over the top. The issue I have is with my HTML files.
According to CloudFront's FAQ:
Amazon CloudFront uses these cache control ...
2
votes
2answers
193 views
Amazon EC2 - What's the point of having Instance storage if it is non-permanent and EBS provides a permanent store?
I'm specifically looking at the Standard Large On-Demand instance which has 850 GB of instance storage.
What I really don't understand is why have 850 GB of storage on the instance if that data will ...
-2
votes
3answers
154 views
amazon s3 vs amazon ec2 regarding cost, and which better for image hosting website? [closed]
I want to know which will cost more, s3 or ec2.
I need to know by example to estimate my future payments.
Also which of them will be better for an image hosting website?
2
votes
3answers
392 views
Uploading 200GB of files to S3
What's the best way to upload 200GB tar.gz files to S3 in Linux? On researching I found that S3 limit on objects has been increased to 5TB and came to know about the multi part upload mechanism to ...
