What's your threat model?
Good guy accidentally destroys git repository or other data: Make sure your users can only write to their homedir, tmp and the git repository. Back up the repository after every commit and practice recovery procedures regularly.
Bad guy steals someone's private key and wants to own your system: Your attack surface is limited to parts of sshd protected by authentication - and whatever shell interface you use, so make it as small as possible. Python/Ruby wrappers involve a lot of code, and a lot of room for errors. Use the git-shell or consider using the Match and ChrootDirectory directives in sshd_config to limit which parts of the filesystem the users can access.
Bad guy steals someone's private key and gets your source code: Teach users how to protect their private keys and establish procedures for revoking keys fast (e.g. who to call at 4am to have keys revoked).
.ssh/authorized_keys, instead use the
AuthorizedKeyfile keyword in sshd_config to specify a path were users don't have write access, e.g.
/etc/ssh/keys/%u. It's too hard to make sure users will never be able to overwrite .ssh/authorized_keys with their own copy.
Also, keep the user keyfiles and sshd_config version-controlled as to allow audits.