I have a server thats primarily used for SRCDS, I host a few Team Fortress 2 servers on it and I would much prefer a ssh-like (more so the no-gui part) for something like this, mostly because I can't navigate with my phone on VNC very well and also because there really is no need for a gui in this situation. I sort of want to make the switch but I am unsure, because my server 2003 is very fast and If I do need to reboot, it can reboot in under 30 seconds to desktop. Should I switch?
No, what warren suggested is sound advice. Cygwin would be the way to go. There's no reason to go through the pain of setting everything up again on a new operating system unless you have other reasons such as: wanting to learn linux, wanting to use linux to lower your per-month server costs (assuming you don't own the Win2003 server license outright) etc.
If your only reason for switching is because you want ssh access, install cygwin.
If your reason for switching involves being more experienced running one over the other, then maybe.
But as asked - no: you shouldn't switch just for the sake of switching.