Larger websites and services, such as Facebook, often use a Content Delivery Network (CDN) to route your request to the closest (and therefore likely fastest) server for their content. They do this via DNS anycast routing, by returning the closest server's address when you perform a DNS query via your DNS server. In other words, where your DNS server is located physically can have an effect on the speeds and response times you get from certain CDN's. Use a DNS server that's close by.
However, these public DNS servers use anycast routing to provide your with a DNS response from the nearest server. Otherwise they couldn't provide you with such fast responses and high uptimes. For example, when you query 184.108.40.206 from The Netherlands, the server that answers the query is not the same one as when you query from Japan. This may partially compensate the CDN problem.
Google itself has a warning to the same effect (where resolver is the DNS server):
Note, however, that because nameservers geolocate according to the resolver's IP address rather than the user's, Google Public DNS has the same limitations as other open DNS services: that is, the server to which a user is referred might be farther away than one to which a local DNS provider would have referred. This could cause a slower browsing experience for certain sites.
I couldn't find a list of Google's public DNS locations, but OpenDNS has a list of cities on their status page that should give you an idea which one you're closest too.