In addition to what Evan stated in his answer, do you in fact have networks of 10.54.x.x, 10.16.x.x, and 10.32.x.x separated by a router or routers? It sounds like maybe you have simply set up multiple scopes for clients that are all on the same network. Is that the case?
In addition, (as Evan stated) DHCP broadcast messages from clients on a different network than the DHCP server need to be "relayed" to the server via a DHCP relay agent (usually configured on the router itself). The DHCP relay agent transforms the broadcast messages into unicast messages, so you should not see any broadcast messages reaching the server from networks other than the DHCP server network itself. You should be seeing unicast messages from the DHCP relay agent for clients on different networks. Is that what you're seeing?