Some providers will park your name for free (they almost always put advertising up on your sites). Some charge a small fee to put up an "under construction" or similar page.
Otherwise you just have to get regular hosting like any other site. A very basic plan will do if you have only one "under construction" or "coming soon" page to host.
If your interested in running your own servers (which is not clear by the title), I recommend installing Bind9 for Windows, available for free from the ISC.
Bind is the most widely supported free DNS server, and there are many tutorials that are pretty easy to understand (though most will have to be adapted, as they're written for *nix). The server will come with sample configuration files, the complete Admin Reference Manual, and supporting tools for diagnosing problems (it's a pretty well rounded download).