I have almost the exact same requirements as you and I did a lot of research into this topic. You can definitely use Puppet/Chef but the learning curve is high. Also, very few recipes for Puppet/Chef I've seen actually fully support Windows so expect having to write your own for Windows that would otherwise be available for Linux.
For Linux management, Ansible will fulfill all your needs. It's extremely easy to use, install, and doesn't require a central server. I have had success using Ansible with Windows by installing Cygwin with OpenSSH. You could automate the Cygwin install and configuration w/ some simple batch/PS scripts in order to bootstrap your server. You then will be able to use Ansible with it as long as it's running Python from the Cygwin install. Windows isn't really supported on Ansible and there may bugs of course but I've been able to use it successfully. If you are a dev, it's even fairly easy to understand Ansible's source code incase you have any issues. The devs are also super responsive when it comes to bugs. I submitted a bug I found related to paths and it was fixed in a day. You will definitely come across some issues in Windows but I think at the end of the day it's still easier assuming you don't have the time and money to invest in Chef/Puppet.
Edit: It looks like SaltStack now has support for Windows. It seems to work with YAML files which is similar to Ansible. Definitely worth checking out now for Windows.