I am setting up a Windows 2003 machine to host a website, are there any settings that I should pay special attention to in regards to security?
In other words, is there dangerous default setting I should know about?
The NSA puts out a comprehensive guide to securing operating systems. While not every option will be required for your environment, the explanations about each change are helpful in understanding the reasons for it. NSA - Operating Systems Guide
You can also check out the SANS Reading Room for papers written on a broader range of topics, but primarily with a security focus.
Microsoft have excellent baseline security checklists, and there are compliance tools.
The one for Windows 2003 is here