I've found all sorts of documentation on what IPSEC is and how to (in very non-specific ways) configure it, but I can't seem to configure a Windows Firewall With Advanced Security rule that both A) requires all SQL Server traffic to be encrypted, and B) works.
As an exercise, I created an inbound rule that allows all traffic the default MSSQLSERVER service when it is connected to the domain, and tested that I can connect to the SQL Server instance with that rule.
When I change the Action from "Allow the connection" to "Allow the connection if it is secure", using the "Require the connections to be encrypted" option, I can no longer establish a connection. I can't seem to find any information on how to instruct the client to use that encrypted connection, or whether there are other prerequisites that must be met. I also can't seem to find much on using IPSEC with SQL Server in general.
I know how to encrypt SQL Server connections via SSL, but the primary application that uses this server will not connect using SSL until some future release.
How do I get the client operating system to connect using IPSEC from the Windows Firewall with Advanced Security settings in Windows 7?