A DC shouldn't be multihomed (= placed on two different networks at the same time), because this could mess up quite some things unless you can deal properly with DNS registrations and some other settings; and, even in this case, it could mess up some things anyway.
That said, placing it on a public network looks even a worse choice than simply multihoming it...
What is the specific reason for this setup?
About your problem: I don't think there's a way to solve that, other than manually configuring Windows Firewall to block everything you don't need on the public side and allow everything you need on the private side; Windows really doesn't have any way to read your mind and say "hey, this is my private network and this is a public one I shouldn't trust": it's a domain controller, obviously any network connections it could have has to be a domain one. After all, you could very well be creating a domain on a set of public IP addresses, couldn't you?