A HTTP connection is just a TCP connection at the end of the day. So the above is actually fine for HTTP connections (i.e. if you connect to www.a.com, you are actually connecting to an IP address and not www.a.com, so it's actually fine and expected for that connection to be able to address requests for www.b.com).
For HTTPS, like you are saying you are using, it's a little less clear cut...
For HTTP/1.1, there is nothing in HTTP Over TLS RFC nor the Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing RFC, to state how this should be handled, as SNI is a non-mandatory extension to HTTP added after (in fact, as an aside this was one way of getting around clients - like IE8 on XP - if the same cert is used for both virtual hosts, then the appropriate connection could be used after the TLS session was negotiated based on that cert as provided by the default host). However the SNI RFC clearly states that the server MUST check this is the same, so when a server name is given and is different to the host, what you are observing is incorrect (thanks to Michael Hampton for pointing this out as I missed this originally).
HTTP/2 is more clear in this and in fact does allow connection reuse in certain conditions and has this to say:
A connection can be reused as long as the origin server is
authoritative (Section 10.1). For TCP connections without TLS, this
depends on the host having resolved to the same IP address.
For https resources, connection reuse additionally depends on having a
certificate that is valid for the host in the URI. The certificate
presented by the server MUST satisfy any checks that the client would
perform when forming a new TLS connection for the host in the URI.
An origin server might offer a certificate with multiple
subjectAltName attributes or names with wildcards, one of which is
valid for the authority in the URI. For example, a certificate with a
subjectAltName of *.example.com might permit the use of the same
connection for requests to URIs starting with https://a.example.com/
So if a.crt also includes include www.b.com in the Subject Alternative Name field then above is actually fine under HTTP/2, though not under HTTP/1.1.
However, just to be sure, I have repeated your observations when a.crt definitely does NOT cover www.b.com and and that does indeed sound like a bug and also potentially a security risk, as a request for one virtual host could be read by another virtual host (despite not having access to the key that message was decrypted with) if someone was able to inject an invalid host header (which thankfully browsers do not make easy to do - so admittedly I cannot demonstrate this). This could leak cookies or other information to the secondary server.
I have raised this to the nginx security team who say that it is not a security flaw with nginx, and that it is up to the client to validate certificates, and then not to send malformed requests for hosts that they should be accessing over that connection. I strongly disagree with this,and say that the whole point of HTTPS is to only allow valid endpoints to unencrypt messages! So I think the check should be on both the client and the server side and in fact this exact scenario was discussed by the TLS working group at that time that SNI was specific, which is why above wording was added to the RFC. Additionally, the nginx security team say that in this scenario nginx should be set up with a joint certificate for both hosts (which makes no sense to me as severely limits the usefulness of virtual hosting in nginx!), but if a nginx server operator really wants to restrict this, then they can do by checking the
$ssl_server_name variable (which to me means it's a relatively easy check that nginx could do by default!). Anyway it's their software and so their entitled to their opinion but I disagree.
It’s also been pointed out that this technique is sometimes used for avoiding censorship in a process known as Domain Fronting. Which is fair enough but which I think should be explicitly enabled rather than be allowed by default.
Incidentally Apache does not do the same and (correctly IMHO) returns a "400 Bad Request" prior to 2.4.17 and the new "421 (Misdirected Request)" after 2.4.17 - which was a new HTTP status code created especially for this scenario.