how can I find out (using windows or internet) if xyz (in particular NS1.XYZNAMESERVER.COM) is valid DNS server?
dig example.com @ns-address host example.com ns-address
nslookup example.com ns-address
If the server is not a DNS server, you will get "No response from server" or "Timeout".
Try connecting to it:
53/TCP,UDP Domain Name System (DNS) Official
Port 53, TCP.
Next steop would be asking it whether it knows something... and thus can handle the TCP protocol
Command line alternative: script something using nslookup.
Don't - if you don't know it is a trusted DNS server, don't use it. It may LOOK like a DNS server, in terms of giving answers to DNS queries, but you should not trust that those are the correct answers.
Of course you might be hunting DNS services for a whole other reason :)
Finding out if it's a valid DNS server is easy, just try querying it.
Finding out if it's authorative and/or returning proper results for the domains in question is harder, depending on your level of trust in things.
Ultimately you would start querying one of the known root name servers for the top level domain and go from there until you get where you want... why? Well, how do you know that the name server you're currently querying can be trusted? ^^
Also see this SO question.