Don't try to load balance or otherwise "HA" your time servers. That goes contrary to how NTP is designed to work -- by comparing the servers against each other. DNS based failover is a non-starter, too. There is also no point in a naive health check of ntpd, because the daemon can be running and happy, despite having lost its mind and giving out invalid time signals.
Note that ntpd will resolve the name of a designated time server into an IP address when it starts, and will not attempt to re-resolve that information once it is running (unless/until you tell it to re-read/reload the configuration file).
This means that if you think you're getting higher reliability by having just one server name but multiple IP addresses (which might point to multiple machines), you should think again. When you're managing a very large group of NTP servers (e.g., pool.ntp.org) there are advantages to this, but for anything smaller, you're almost certainly not going to get the behaviour you're probably thinking about.
Also, don't try to give multiple machines the same IP address, or hide them behind a load-balancing device, either - that will really confuse ntpd and make the situation far worse.
It's much better to have a specified set of servers, each with their own unique IP address, and configure the clients to connect to multiple servers in this set and then let the clients deal with the issues of what happens when one or more of their servers becomes unreachable or unreliable.
Configure the servers individually and set the clients to use all of them... all 4 of them, since 4 is considered the low bound for an appropriate number of servers, as it leaves you with what is still a high quality time signal from three servers, if one of the servers should lose its mind. One bad out of three doesn't allow the one bad server to be effectively ignored.
From my perspective, the NAT instances used in VPC are appropriate candidates for running your time servers.
Or... consider that once ntpd stabilizes, it generates less and less traffic, and don't worry too much about the relatively negligible amount of Internet traffic (but of course don't configure the servers all to connect directly to stratum 1 sources).