My understanding of your solution leads me to think that you wish to use the same ubuntu server you already have to also perform the transcoding of your uploaded media files.
The problem is objects stored in S3 are not accessible like files on a normal file system, you could download the file from S3 and process it on the web server instance however this would be a complicated and potentially inefficient setup.
It would be better to decouple the transcoding process from your ubuntu server and let it happen in true cloud style. The best solution for this is S3 events and Lambda.
So based on my understanding of your use case here is what I would recommend for your solution:
- Create a bucket with the appropriate permissions for receiving files (best not to make this public or your bills could get quite expensive)
- Create an S3 event to trigger Lambda when a object is created / put / updated (depending on what you wish to trigger off and how files are placed on S3)
- Use lambda to process your file, or have Lambda start an instance that has a UserData script that will do the work.
Regarding the instance's UserData, here is the steps you may wish to have it undertake:
- download ffmpeg
- install ffmpeg
- download from s3 the file (you can pass this argument from lambda)
- process the file using ffmpeg
- upload the file to s3
- use the ec2 metadata service to lookup the instances own instance id
- run the aws cli command for terminating an instance supplying the resolved instance id.
You could be better off creating an AMI that already has ffmpeg installed but you would need to decide if its cheaper to spend 5 minutes syspreping each instance or paying for an AMI to always be on hand. I would say though unless your processing takes longer than an hour or your use case need the file to be returned ASAP, you would be better off installing ffmpeg each time as AWS bills for full hours even if you only use 15 minutes.
Recommendations for this approach:
Chances are you may wish to undertake further activities when the newly processed file is created, so why not make use of s3 events to fire off another Lambda process? :)
Also to help keep things clean, if your solution allows for it, try to upload your created files to s3 under a different key path to where you place your uploaded files.
Alternate Option: Use Elastic Transcoder
An alternate option is to make use of the AWS Elastic Transcoder service. You would send jobs to the transcoder in the same way by triggering lambda when a S3 bucket is updated and having it process the files in that bucket. Elastic Transcoder can then notify an SNS queue which then can trigger an email or another Lambda query to deal with the created file.
Elastic Transcoder would be a cooler and likely better approach but it will require a bit more work.
An Alternate Alternate Solution: Use Amazon SQS
If you want the processing to occur on the ubuntu instance rather than spinning up another instance from Lambda, you could use S3 events to trigger lambda to publish a job to Amazon SQS.
Amazon SQS will then let an agent of your own creation poll Amazon SQS for jobs which will then let you reference a file in s3 needing transcoding. This is rather complicated though and I only include it for completeness and unless you really do need to perform this work on the ubuntu instance you already have.