Is this how it works?
What would happen if the server just responded with syn
Simultaneous initiation is described in RFC 793 section 3.4 and also covered by RFC 1122 section 22.214.171.124.
The procedure also works if two TCP
simultaneously initiate the procedure. When simultaneous attempt
occurs, each TCP receives a "SYN" segment which carries no
acknowledgment after it has sent a "SYN".
or with just ack?
I'm not entirely sure, but I think a RST is sent back and the connection is terminated.
What would happen if the server doesn't respond at all?
What would happen if the client doesn't respond with ack?
When no response is received, the client remains in SYN-SENT (server - SYN-RECEIVED) state and eventually times out.
Overall, RFC 793 "Transmission Control Protocol" is an excellent resource.
See also RFC 1122 "Requirements for Internet Hosts -- Communication Layers".