I tried 64 bit with 8.04. There were terrible problems with triangle of Web applications, Java and Firefox. Some folks said they fixed it, I tried all the fixes and never could get a Java app to launch from Firefox (64 or 32 bit firefox and/or Java). Meanwhile I had no problems with 32 bit 8.04 on another system.
Maybe this is all fixed, or was all fixed, or maybe there was an issue with my very early 64 bit hardware (socket 754), or maybe me. I've been running 64 bit Linux Red Hat servers for 5 years now with no problems running 32 bit java apps under Firefox. It was enough that when I updated to 9.04 on that system I went back to 32 bit.
Hopefully this issue has been fixed, and by fixed, I do not mean an elaborate work-around as published for 8.04 64bit, but fixed on install.
I would not hesitate to use 64bit for a server.