First off, you have two conflicting requirements: you don't want a do-it-yourself approach, but you want it cheap. You need to figure out which of those two is more important.
What's this server needed for?
If it's feasible to host it off-site, you could look at leasing a box at a datacenter (Rackspace, Cari, etc.) You could even get a fully managed server so you only have to worry about the application itself. Of course this extra service is more costly. For the super-cheap start-up route, you could even use Amazon EC2. However, EC2 is going to be more expensive in the long run and you're going to get yoru hands dirty managing it.
If it's not feasible to host it offsite (it's also functioning as a file server for the office, etc) then yes, buying a server from Dell (or similar) would be your best bet.
The cheapest that you'll likely be able to find a server for sale is from your local off-lease reseller, Craigslist, or Ebay. You can find really nice, recently decommissioned servers for awesome prices. Be aware though that they don't normally come with support though, so once again you'll be on the hook for that.
Edit: If you're purchasing used gear, be sure that everything is in order with the OS licensing. If the OS came with the hardware (OEM version), you should be ok. However, larger corporations will sometimes purchase the OS separately from the hardware (MS Open Licensing). That type of license would not be transferable to you when you purchase the box.