The upside is that you stop everything which cannot adapt (spam email malware, etc) and everyone who doesn't know how to get around it.
The other answers here seem to be of the nature "whatever you do I will get around it so don't even bother trying", but the majority of people do not know how to do things to get around firewall restrictions so you reduce the risk of trouble a lot.
On top of that, if someone can get to a service outside your firewall first try because the port is open, that's one thing, but if they have to deliberately work their way around your firewall that's quite another. It might give you more evidence to demonstrate misuse of computer systems, for instance.
And for regulatory compliance or auditing reasons you might need to justify every port open on your firewall, it would be better to be able to say "Port 1433 outbound because accounts need it for program Q even though it might open the door to some misuse" than "SQL works because we haven't bothered blocking anything".
tl;dr: Security isn't about one silver bullet solution which fixes anything, and option 2 can be another layer in helping to reduce risk and misuse.