I've found several firmwares out there: OpenWRT, DD-WRT, Tomato and Oleg's. I'm wondering what is the best choice?
Router is working in the home network (desktop through wires, laptop uses WiFi). Also I plan to run torrent and VPN client on it.
OpenWRT.org This is definitely something where everyone will just recommend whichever one they're familiar with. But please note that most of these are based on openwrt, and largely the forks were created because at that time openwrt did not have a very good web interface. These days openwrt does have a good web interface, and I highly recommend it.
See this example of how openwrt makes everything easy :)
I've had no problem with Tomato. I don't run any services on it, but I don't see any reason you couldn't. There is command-line access; there are startup scripts; there is SMB mounting; there is a flash filesystem.
I don't know if you already have a device, but make sure that the software you choose supports the hardware you have or are going to get.
I have Asus WL-500g and I use DD-WRT. I currently have v24 SP1 installed and there are no issues with it.
I find it really easy to configure and it supports Asus hardware quite well. Torrenting from it is little bit harder with DD-WRT, but there is support for installing additional packages.
VPN is supported just fine.