Use following in squid A configuration (normally /etc/squid/squid.conf)
cache_peer <squid server B IP> parent <server B squid port> 3130 no-query no-digest no-netdb-exchange
acl otherproxy url_regex "/etc/squid/divert.txt"
cache_peer_access <Squid server B IP> allow otherproxy
never_direct allow otherproxy
cache_peer_access <squid server B IP> deny all
And following in squid B configuration at top of other acls
acl A_proxy src <Squid server A IP>/32
http_access allow A_proxy
After that you can add regular expressions to file /etc/squid/divert.txt and reload squid. The matching URLs will be requested from Server B and not directly.
The reason why I mentioned adding acl on proxy B too is too avoid authentication, connection limit, etc. settings on proxy B affecting clients of proxy A as such problems are very hard to debug. So it is better to allow all requests from proxy A to proxy B. If you want to block something block it in both proxy configurations.