Just in case anyone else has this problem - I've finally managed to fix it.
Someone had patched our 4G backup router into the SCCM build VLAN – this meant that it was assigning
external IP addresses to the clients via DHCP, during the middle of the deployment.
This wasn’t easy to pick up as A. it was intermittent and B. it didn’t happen
at the boot screen, ...
Have you checked "Enable a PXE responder without Windows Deployment Service" in the distribution point settings?
If not, this should be the "Windows Deployment Services Server" service on the distribution point server.
If you enabled this option. I think the correct service is the "ConfigMgr PXE Responder service".