I occasionally look at sites like netcraft and am curious if there is an unobtrusive way to ping a server and see what publicly facing software it is running? Are sites like netcraft using some sophisticated heuristics to infer their data or can certain kinds of (non-abusive) requests lead to straightforward answers?
NMAP allows you to do OS detection and service identification. I don't know how netcraft does it -- and using nmap could very quickly move out of the 'unobtrusive' category. But you can test it in-house for sure...
I think NetCraft use the Server: header returned by the webserver. No special tools are needed to do this
$ curl -I http://www.microsoft.com | grep "Server:" Server: Microsoft-IIS/7.0 $ curl -I http://www.apache.org | grep "Server:" Server: Apache/2.2.9 (Unix)
It's based on the ICMP fingerprint in the packets that you get back (if you get them back). Different IP stacks reply different to echo's and that gives you a hint to what is on the inside. Check this out.