Many DNS servers are pre-configured with version information in DNS TXT records for the version.bind
label in the CHAOS class. If this is the case with yours as well, you could retreive it by running
dig @dns.name.server version.bind chaos txt
Typical answers might include
;; ANSWER SECTION:
version.bind. 0 CH TXT "9.8.1-P1"
or
;; ANSWER SECTION:
version.bind. 1476526080 IN TXT "Microsoft DNS 6.1.7600 (1DB04228)"
or
;; ANSWER SECTION:
version.bind. 0 CH TXT "dnsmasq-2.47"
Apart from this, tools for DNS fingerprinting exist, fpdns probably being the most advanced one.