Oh my... If you have 60Gb of RAM, please save yourself the hassle and use a 64-bit os. Also, 60 SQL instances sounds like a very odd way of doing things and I can't in good conscience suggest it. The amount of money a server of that stature costs, surely a proper SQL license would be in the budget?
Windows is also incredibly intelligent when it comes to memory usage. It will release (page) memory from as required. I've seen SQL Server consume 95% of the avaliable memory in a server, and the other services on the server were sidelined as appropriate.
-- Update --
Judging by your comments, what makes you think that 64-bit windows is eating your RAM? If you're looking at the "Free" reading in the "Physical Memory" group in the Task Manager, this is not an appropriate place to be checking.
-- Update --
Regarding more of your comments below, what you really want to know is: Which is more efficient - 1x40Gb process, or 40x1Gb processes, and this question can't be answered easilly.
It's a bit like this: You need to transport 40 people. Do you buy 40x VW Polo's, or do you buy a 40-seater bus? They both cost about the same. It really depends on what you want to do. Is everyone going to the same place? Do they need to get there at the same speed? Do they need comfort? Do they need safety? Do they hate eachother and are they going to stab eachother in the face at the mere sight of eachother?
Applied to your scenario of SQL server, and a few assumptions (based on other questions you've asked on ServerFault) I can categorically say that you want one massive SQL Server instance consuming as much RAM as it can get its greedy little hands on.