This should be actually a very obvious setup, but somehow I have missed the obvious step!
Default assumptions: FreeBSD 6.4-RELEASE-p11, Squid 2.7.STABLE9. These are not changeable, so "upgrade to the latest version" does not apply for me. FreeBSD is using ipfw (also not changeable).
Our setup works the following way. The jail host does nothing but run ipfw. One special jail is configured with a valid IP address (the rest have private addresses). This jail runs Squid to forward incoming requests and distributes it among the many other jails where Apache is running. This is easily accomplished with Squid in accel mode (with http_port [real IP address]:80 vhost), using squirm to properly redirect URLs to the appropriate jail, and making sure /etc/hosts on the host knows which jail is pointed to. ipfw restricts incoming requests, and allows jails to respond. This works flawlessly, and has been working for years without a glitch.
However, an increasing number of hosted websites also require contacting external webservices. Since all websites have to be on the jails, the only solution is to run Squid also as a proxy server for them (and only for them), and allow outgoing connections only to the required services (typical examples: Yahoo! weather, or APIs from Google), which required setting them up on squid.conf as well as allowing outgoing connections from the jails. So website jails can only connect to the external world via the Squid jail; there is deliberately no "workaround" for security reasons (that part is also not changeable).
This also works well, and has been working for several years, also without a glitch.
Unfortunately for our beautiful setup, an increasing number of web services are (very thoughtfully) moving to HTTPS for extra security. Here is the catch. Even though Squid's configuration is prepared to allow outgoing HTTPS requests through itself, the result is always a 400 Bad Request, which shows up on the logs as:
1345810897.782 0 <internal jail address> TCP_DENIED/400 1695 CONNECT :0 - NONE/- text/html
Note the strange "CONNECT :0" request. Other links on this topic point to a badly configured port translation/connection configuration somewhere, but this is not the case; the error is exactly the same, no matter what client is used (lynx, curl, wget, own XML-RPC libraries, etc.).
The jail host configuration, even though severely limited with ipfw, has absolutely no problem with outgoing connections for HTTPS requests.
The "special" jail where Squid runs has also no problems with outgoing HTTPS requests, so I know that the firewall using ipfw is not to be blamed. However, an HTTPS request made on that jail through Squid will fail exactly like any requests from the other jails.
What I believe that is happening is that this kind of solution will not work for HTTPS, since all that gets relayed through Squid is the initial CONNECT — the rest of the data is "invisible" to Squid. Under normal circumstances, NAT on a router should handle the return of the data correctly. But this special configuration doesn't really rely on NAT, and I'm quite sure that enabling NAT would also be forbidden — the admin doesn't want the internal IP addresses to be exposed publicly via NAT. So Squid has to handle all the conversions and all the connections.
Does this make sense? I realise that HTTPS is quite a different protocol than HTTP, and, as such, some things which are easy under HTTP might not work under HTTPS because of that. There are also trade-offs to be taken into account: a complete revamp of the whole solution costs labour and time, and it might not justify that just to give access to a very few webservices now and then.
Note that the above complex setup of jails is not changeable (e.g. running squid on the real host instead of the special Squid jail is not an option). All that is allowed is tweaking ipfw and configuring Squid. Extra software that might be required is allowed in extreme cases. Ugrading/changing existing software is not.
Thanks in advance. My apologies for the long email and the relative lack of configuration dumps; this is a relatively mission-critical system and I'm not allowed to post much of its configuration publicly. However, I will be more than glad to answer yes/no questions like "Did you remember to set [obscure configuration command] to 'true'?"