I have set up a local DNS server on a Windows Server 2008 machine. When visiting websites I see the results are filling up the server's DNS cache. But when using the benchmark tool namebench I see that my server's mean responds time is almost 4 times slower in comparison with my default ISP DNS server.
Mean Response Duration

Fastest Individual Response Duration

Response Distribution Chart (First 200ms)

Is there a way I could speed up or optimize the DNS server?