When this question was asked, there probably was no TCP SYN proxy with SYN cookies that was open source (there probably was OpenBSD's PF but it uses SYN cache instead of SYN cookies). Now, however, there is the netfilter SYN proxy: https://lwn.net/Articles/563151/
As an alternative to this netfilter SYN proxy, you can code your own. I have done that. My implementation is 9000 lines of main component code and 17 000 lines of supporting library code written in the C language. The line count includes very thorough unit tests. I won't recommend this "code your own" approach for production purposes, but if you have some spare time, coding your own SYN proxy may be a good way to learn a lot about TCP. I did my implementation in a bit over a month.
The netfilter SYN proxy apparently some time ago had a bug where TCP RST packet was not properly modified to be accepted by the host that opened the connection. So, if you managed to proxy the connection to the SYN proxy, but the other endpoint didn't accept it, the connection wasn't properly closed so that the initiating endpoint would notice it. I'm not sure if they have fixed the bug now.
As for my recommendation, I would recommend running endpoint hosts that have TCP SYN cookie support. That makes SYN proxy obsolete. If you absolutely have to run endpoint hosts that have no SYN cookie support, then SYN proxy may be a good idea. Also, if you run a firewall, it creates a new target for resource exhaustion: don't exhaust the resources of the endpoint host but instead exhaust the resources of the firewall. A firewall has no other way of protecting itself than a proper SYN proxy with SYN cookies. So, if using a firewall, do consider ones that have TCP SYN proxy support. OpenBSD's PF has SYN proxy with SYN cache that is vulnerable to an attack by resource exhaustion albeit not as vulnerable as a firewall not having SYN cache. Linux's iptables can be configured with the SYN proxy module in a manner that uses SYN cookies and thus is not vulnerable to an attack by resource exhaustion.