The biggest question you need to address is how you're going to get support for the system. There are vendors out there producing pre-configured hardware and providing support, SuSE and RedHat both offer support packages (OK, Ubuntu does too). You're probably going to need more hand-holding than the odd post on ServerFault. There's probably a Linux Users Group somewhere near you - go find them and talk to them.
I started off using RedHat distributions, then switched to SuSE due to ease of management. But then stopped using SuSE when it became rather bloated and slow (I've not revisited this ifor about 5 years). I've also used Mandriva, PCLinuxOS, Knoppix, Debian, Gentoo and others. Ubuntu is my least favourite distribution - it just tries to be so different from a standard Unix system. Currently my home system runs Fedora (the community/bleeding edge version of RedHat). Note that although you can't get 'free' RedHat distributions anymore, since its all GPL software, RedHat can't stop other people from compiling their code! You might want to have a look at CentOS - which is essentially RedHat Enterprise Linux.
Regarding hardware, Dells are certainly cheap, IBM don't make them any more. My Own preference for branded boxes is HP - but currently HP do not seem to ship boxes pre-configured with Linux.
If you're thinking of DELL, then I'd recommend a proper server (poweredge) over a desktop machine - they do different jobs - and there's no point in paying for stuff you don't need - like a fast graphics card. Having said that, I see that Dells cheapest server is approx 3 x the price of their cheapest desktop (and running a cluster is always more fun than a single server!)