Just a simple question but I can't find a definitive answer online. Of course cloudfront is good for serving content globally, but is the infrastructure there inherently different from s3, so tha even if the distance to transfer is the same, cloudfront would genrrally be faster?
Yes. CloudFront has lower latencies than S3 even when the request originates from a non-edge location. Generally, CloudFront responds tens of milliseconds faster. Whether you care about such small differences is a separate question.