Two-factor authentication works by allowing access based on two authentication methods. From a remote system point of view, SSH key files are really only one factor, because the remote system cannot know for certain that a password was used to decrypt the private SSH keyfile.
I recommend you keep a copy of your encrypted private SSH keyfile somewhere safe, like on an SD card at a friends house - on in a bank vault. If you lose your private SSH key, you will need to contact the administrator of the system and supply him with a new public SSH key based on your new private SSH key.
Account recovery of this sort often becomes the biggest security hole. How does the administrator know that you're you, if you don't have you original private SSH key? He would need to know your voice or face or have some other means of verifying your identity.