We have a very complex network boot environment to keep traffic separated for our imaging networks, which process hundred of machines an hour.
We use the WDS Server Selection menu tweak (http://www.stephan-schwarz.net/?p=58) to allow for multiple PXE servers as a manual load balancing solution.
In the process, I found there is a cap of 20 sources for non-UEFI machines and 16 sources for UEFI machines (I posted on Technet regarding this and am content its a BIOS cap)
I've since found the ability of MDT to define a list of server sources with an unknown limit so that's a load balancing solution once booted into the MDT image.
However, very rarely, I see a PXE problem I can't understand.
When I enter into the WDS Server Selection Menu, I see all 16 servers as IP [SERVER NAME (no domain)] as provided by the server that answered first. Policy is to choose a specific server (from 1-16). One of three things happens:
1- The server in question loads (Server 01 loaded the WDS Server Selection Menu, Server 05 was chosen and is booted from). This occurs 90% of the time.
2- The original source server loads instead of the chosen server. This occurs rarely, but may also be hidden by the fact that the server data is almost always the same (up until now)
3- The machine attempts to load an arbitrary IP, and it could be anything- including public IPs. Rebooting and trying again could cause a different IP, over and over. Sometimes the IP is the same. For example, I captured the packet traffic from a machine in Wireshark, and I see normal PXE traffic until the unit in question sends out a DHCP Request for an odd destination IP like 184.108.40.206
Now none of this happens if I let a machine randomly choose a boot server via network boot. I feel (though haven't tested an incredible amount) that it doesn't happen if the server needed is in the first 8 in UEFI (at least on the device in question, UEFI WDS Server Select shows a few devices and then you need to scroll).
Is there any rationale behind this? I have a 2MB pcap file at https://www.dropbox.com/s/qwsnvgda1lhxbcx/pcap_wds_ip_weirdness.pcapng?dl=0 that contains the traffic leading from fresh boot to the wrong IP being requested. Suggested filter would be eth.src == d4:c9:ef:f8:e4:db || eth.dst == d4:c9:ef:f8:e4:db
The environment is mostly HP - ProCurve 2530G and (rarely) 2510G switches with IGMP and STP enabled, Windows Server 2012 R2 servers on an ESXi Cluster, HP Elitebook 810 G2 devices mainly.
I appreciate any insight.