You shouldn't assign other IP addresses to this adapter. Just the one that's used to connect to this host. If your router is configured correctly and forwards the traffic (in bridge mode) to the interface of the NIC in your server (all IP addresses), then assigning the correct external IP address to a virtual machine's vNIC (inside the VM) should do the trick.
So, remove all other IPs from the host NIC (they are useless there), leave only the one you want to use to connect to this host. Assign other IP addresses directly to Virtual Machines.