4

What is the default way to upload files from a EC2 web server to a S3 bucket for content delivery?

Improve this question

6 Answers 6

Reset to default
3

The answer is rather dependent on the purpose. For instance, if your EC2 instance is running a PHP application, then using the PHP-SDK would be the best route. Some applications might find it useful to mount an S3 bucket as a local file-system (e.g. S3-fuse).

If however, you are simply trying to upload a file via the shell from EC2 to S3, I would recommend Tim Kay's aws script. It is a self-contained perl script that can run without installation, or can be installed to setup alias functions. Uploading a file to an existing bucket using this script amounts to the following:

s3put bucket_name/filename.ext /path/to/local/file/name.ext

It supports most of the features of AWS and is sufficiently verbose in its error messages to make debugging a problem fairly easy.

Improve this answer
2

You can use s3cp tool. It's a like of scp command. For download and documentation, see http://www.beaconhill.com/opensource/s3cp.html and http://aws.amazon.com/code/Java/3124

After install, you can for example copy to S3: s3cp local-file s3://bucket/object[/]

Improve this answer
2

Here's a simple example to illustrate uploading directly to s3

http://sente.cc/upload_to_s3.html

code:

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  </head>
  <body>
  <h3>refresh the page after you've submitted to see your new image</h3>
    <div style="width:300px">
    <form action="http://s3.amazonaws.com/dev.sente" method="post" enctype="multipart/form-data">
      <fieldset>
      <input type="hidden" name="acl" value="public-read" /> <br />
      <i>name of key:</i><input type="text" name="key" readonly="true" value="image.jpg" /> <br />
      <input name="file" type="file" /> <br />
      <input name="submit" value="Upload" type="submit" />
    </fieldset>
    </form>
  </div>
    <br>
    <a href="http://s3.amazonaws.com/dev.sente/image.jpg">http://s3.amazonaws.com/dev.sente/image.jpg</a>
      <br>
      <a href="http://s3.amazonaws.com/dev.sente/image.jpg"><img src="http://s3.amazonaws.com/dev.sente/image.jpg"></a>
    </a>
  </body>
</html>

copied from my answer to a similar question - here

Improve this answer
2

Using the aws cli commands, if you are using an EC2 Linux virtual machine and want to move files and not directories :

     aws s3 put <yourfilename> s3://<yourbucketname>/<yourdirectory>
Improve this answer
1

Via HTTP, the same way you upload to S3 from, well, anywhere else.

Improve this answer
7
  • If a script on the EC2 needs to get information about a image stored in the S3, the EC2 would need to request the S3 server through HTTP? Wouldn't that cost bandwidth every request and wouldn't that be slower than having the same image directly on the EC2 and requesting that image instead? Should I keep a copy of the image on the EC2 and in the S3 bucket to reduce requests made by scripts on the EC2 to the S3???
    – John
    Jun 30, 2011 at 20:06
  • Unless I'm mistaken, there are no transfer costs between EC2 and S3, but I may be mistaken on that. Amazon outlines things pretty well in their AWS pricing documentation.
    – EEAA
    Jun 30, 2011 at 20:07
  • Still, wouldn't it be slower to request a image through HTTP on a different server (S3/cloudfront) rather than having the image on the same server (EC2) ?
    – John
    Jun 30, 2011 at 20:11
  • In most cases I've see, the images are served directly out of S3 to the user, not passing through EC2 at all. Doing things that was provides a level of resiliency that EC2 can't provide.
    – EEAA
    Jun 30, 2011 at 20:19
  • Sorry I do not understand. I'll try to clarify my question: Would it be faster or slower have a script on my EC2 web server access a image file that is already on the EC2 sever(locally) or have the EC2 server access the image file on the S3 (which I assume would be remote through HTTP, which also I assume uses bandwidth) ?
    – John
    Jun 30, 2011 at 20:23
1

In general if you're uploading files from an application to S3 for the purpose of setting up a CDN to use for that application you would want to first look and see if that application didn't already have the functionality or a plugin to do that. If you take Wordpress for instance there is the W3 Total Cache plugin that handles this for you very nicely. It is of course written in PHP and using the AWS API calls to handle the uploads/deletions when necessary.

If you're writing your own application then I would look for libraries that assist in communicating with AWS. For Python you would want to look at Boto for instance which allows you to interact with the AWS API seamlessly. As well you can always check out Amazon itself for information on the other SDK APIs that you could use.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .