The most common problem is that your gateway rewrites the destination address of the packet to the internal server, but not the source. So, when the internal server responds it sees that the packet came from something on the local network, sends back the packet directly - and the client can't tell this is from the server, because the packet still has the internal, not the public, address on it.
The standard fix is to force the traffic to come back through your gateway. One way to achieve that is to put the server that the public address redirects to in a "DMZ", so that traffic between the client and the server has to pass through the router.
The other way is to also apply NAT to the source address of internal connections to the external IP, so that they look like they come from the gateway. The internal server will then respond to the gateway, that will undo both NAT changes, and send the packet back to the internal client.