Remember that the more memcached clients you have, the more connections you have to have open. If it's a web service running under Apache pre-forking, and you want to be able to handle 10,000 simultaneous connections, the difference between one server and 4 is 30,000 TCP connections.
As far as performance goes though, it really depends on your application. But having 4 servers may give you 4x as many CPUs and 4x as many network interfaces, unless you really beef up that central server. Above a certain point, the cost of those sort of upgrades goes non-linear (a 32-core system is going to cost way more than 4x the cost of a dual socket quad core, for example). But, I haven't seen memcached be CPU bounded typically, so it's definitely plausible that the single server may not have to be such a high end server.
But, as with many things, it's probably something you have to test for yourself with your specific applications. It seems likely to me that a single 48GB server could handle the load, so try getting a modest server with 48GB of RAM, put munin on it and run some stress testing. If you find bottle-necks, you have more information about whether you need to get 3 more servers and spread the RAM around them, or perhaps take some other action.