This should do it
iptables -I OUTPUT -p tcp --tcp-flags ALL RST,ACK -j DROP
RST/ACK is sent when nobody is listening to the port. But you want to keep RST for legitimate use. You could also use the state flags ESTABLISHED to make sure you allow for RST in that state.
The chain could be FORWARD too, it depends on your setup.