You give one key to the user, and keep one key for yourself. Even if someone intercepts the key the client has, they won't be able to decrypt the information the client has sent back to the server. Note: anyone who intercepts the key the user has will be able to decrypt whatever the server sends to the client - so either you need to work around this (e.g. get the client to generate a new key pair, and send you one or just make sure you never send anything sensitive back). You need to be careful with generating these key pairs, if they're at all predictable, they're as good as no security at all.
To know why this works, check out its Wikipedia page.