Let say I have an django application
Will it be better (performance-wise) to serve the static files with s3 or to setup an nginx server on a ec2 instance?
Nginx is going to be faster for site assets like CSS, nav images, etc. If the client browser already has an open connection for the dynamic page, and keepalives are setup it will use the same connection.
You can now back CloudFront with EC2 hosted content so S3 is no longer a requirement at that level. In addition you can use cloudfront with django generated, but not frequently changing content. You can use nginx's built in caching for semi dynamic content. If you are going to have to run a server for the django portion anyway, it is not much extra work.
If the files you need to host are larger, then S3 starts to look better because the disk performance on smaller EC2 instances is not ideal.
In the end, unless you are dealing with dozens of requests per second, both options are going to be close enough that it will be difficult to measure the differences.
There are many things that can play into this, but if it's an option, I would just serve directly out of S3 if you can. Performance would be at least as good or possibly better than using nginx, and it gives you an easy path to start using AWS's CloudFront CDN in the future if need be.