We are currently hosting a 3TB library of image galleries on our web server, all searchable by users. With about 400 active users at a certain time on the website, our disk reads are pushing 100% I/O utilization, and this looks like it could become a bottleneck. This is all caused by the actual image reads rather than database(which we are caching in RAM using Redis). We don't think using something like Varnish would help us, as the reads are pretty much random - we have about 7 million images, and no groups of them get significantly more reads then others as far as we can tell. We use Cloudflare's free plan, but it doesn't seem to be helping. We simply can't afford a full fledged CDN, as we push about 10TB of bandwidth a month and this would cost thousands of dollars.
Scale horizontally, not vertically. Instead of having one server do all of the work, setup many to do the same. They may not all host the same data, which allows a smaller set to access. Hadoop may be one way to do this. Another could be just in the way it's coded. putting a lot of little servers behind varnish would definitely be something that could help you out.
Even without fancy software, domain sharding is a way to reduce load on a single node: http://mvark.blogspot.com/2009/06/what-is-domain-sharding_17.html