I would like the site to be as fast as possible, are these programs worth installing? How much overhead will they add to my server and how, if at all, will it effect page load times?
webmin only uses resources of the server (cpu and memory) when you call one of the pages. If you rarely use it to make configuration changes, the overhead in general should be able to be neglected. I believe the same is true for virtualmin.
If you're interested in optimising Drupal, and indeed the rest of the stack, then you really need commandline access and to understand what you're changing and not just a gui. Webmin won't add much overhead per se but it will restrict you and won't encourage you to think outside the box.
If this is a server running one or two Drupal sites then you should seriously consider Pantheon if page load times and number of supported concurrent users are a major issue. It doesn't handle several Drupal sites running on the same box, but then if you're really serious about optimisation then that shouldn't bother you too much.
This is kind of an old question, but I know a lot about the subject (I'm one of the developers on both Webmin and Virtualmin), so I thought I'd chime in:
Webmin uses about 10-12MB of memory on a 32 bit system, and about 50% more than that on a 64 bit system (this cost is paid for all software, but particularly for software written in dynamic languages like Perl, Python, PHP, and Ruby). CPU usage for one user is negligible and can be pretty much ignored. If you enable resource data collection (which I think currently only happens if the Webmin server is being managed by Cloudmin), it'll have a cronjob that eats a bit of memory and CPU every few minutes, but should also be pretty minimal.
Virtualmin is another story, and answering the question is more complex. In its default configuration, on a system with more than 512MB of RAM, Virtualmin will use about 100MB (more on a 64 bit system), as it will cache a number of libraries and data structures for performance. Because Virtualmin abstracts dozens of services (Apache, BIND, Postfix, MySQL, monitoring, resource usage, stats, users, groups, applications, etc.) into a single "virtual server" and strays from Webmin's model of only ever editing one configuration file at a time, generating a lot of its pages is much more resource intensive...thus the caching. This is configurable, and there is complete documentation on tuning Virtualmin for low memory systems in the Virtualmin.com documentation, including turning off caching and making Virtualmin take the same ~10MB as Webmin. In recent versions of Webmin, the performance difference is actually quite small if the system is not used by more than one user.
But, it's hard to accurately gauge memory or resource usage in a system running many services. Shared libraries may mean that memory usage of a specific application could be much lower than it appears.
Anyway, we run Drupal for Virtualmin.com, and obviously we also run Webmin, Virtualmin, and Cloudmin on the same system. I worry about performance quite a lot these days, as our system is under pretty significant load most weekdays, but Webmin is never where I look for tuning opportunities. There's always something more pressing and more demanding of resources when I dig down to see what is slowing us down.
So, I'd advise you to not try to guess at what might be hindering performance, and instead look at the usage data, and find the actual resource users and try to alleviate those issues. (Virtualmin can actually help there, though I don't think Virtualmin GPL has the resource monitoring and charting yet).
Hope this helps. If you you do have problems with Webmin or Virtualmin, we're always happy to help. But I doubt your Drupal performance issues have anything to do with Webmin or Virtualmin.