I find it hard to believe that I wouldn't be charged extra for such bandwidth usage.
40 Gbit/sec isn't "usage" -- it's available capacity. You aren't charged for available capacity. You are charged for usage, which is the "Regional Data Transfer Out to Internet (per GB)" mentioned in the CloudFront pricing table. There is no charge for the bandwidth from S3 to CloudFront, when CloudFront fetches objects from S3.
Also, I doubt the S3 servers/nodes could provide 40Gbps connectivity.
Well... don't. :) With one of the large EC2 instance types (e.g. hi1.4xlarge, cc2.8xlarge), if the instance is in the same region as the bucket, and you download a large file from S3, you can easily see downloads of individual files with transfer rates in the 1 Gbit/sec range from S3. It has the capacity.
40 Gbps isn't the capacity available for a single download, of course, but on the scale of AWS, this much bandwidth available it is not at all out of the question. None of the components in the architecture you describe is dedicated to you.
It's shared by all -- or a significant subset of -- customers, so there is no need for 40 Gbit/sec of anything to be dedicated to your use. It also does not need to be in one place. There are over 60 distinct, independent locations on the CloudFront edge network and each browser connects to what CloudFront (with help from Route 53, even if you aren't a Route 53 customer) considers to be the optimal edge location for each browser, using their own proprietary geo-routing database, which appears to be adaptive. (My test site shows which edge location your browser is currently using, if my experimental mapping logic can determine it -- it usually can -- and I see it periodically shift even when I hit the site from home.)
With only 40 edge locations, less than the current number, a comically small amount of bandwidth available at each edge would accommodate 40 Gbit/sec of transfer to a globally-dispersed audience.
Each region's S3 infrastructure is also distributed across hardware spanning the region's availability zones.
40 Gbps in fact isn't even a physical limitation...
And how and when would that be limited?
...it's a software-enforced protective control. Just like the limit of 100,000 requests per second. The request would be denied.
LimitExceeded – The request was denied because a CloudFront limit was exceeded.
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html
Contrast this to CapacityExceeded which would be logged in the event of an actual problem in the CloudFront network preventing the request from being processed.
S3 documentation doesn't mention a bandwidth limit, but among the discussion of scaling:
if you expect a rapid increase in the request rate for a bucket to more than 300 PUT/LIST/DELETE requests per second or more than 800 GET requests per second, we recommend that you open a support case to prepare for the workload and avoid any temporary limits on your request rate.
http://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-considerations.html
Note that this refers to "a rapid increase," implying that a gradual increase would automatically be handled by S3's automatic scaling and/or sharding of its lexically sorted object key index.
What would be the maximum streaming bandwidth using (Elastic Transcoder +) Amazon S3 + CloudFront?
Everything is of course finite, but in the case of the services in question, there is no way to meaningfully describe the limits. It is effectively unlimited for all practical purposes.
Note, for the sake of absurdly playing with numbers, that actually using 40 Gbit/s, all day, all night, for an entire month would almost certainly never happen, but if it did would cost in excess of $400,000 in outbound data transfer (an extremely round number, the regional pricing and discount tiers make the calculation complex) but makes the fact that this much capacity is technically "available" seem a little more believable -- the capacity is there if you want to pay to use it, but if you don't use it, you don't pay for it.
