What goals do you have in mind for your security? Who are you protecting this against?
Broad spectrum, use iptables to firewall port 22 against any unwanted IPs by specifically allowing the IPs you want, then blocking everything else.
You can also specify whether users can use passwords to log in, or whether they have to have certificates, which might be a good idea if you suspect people will try to break into others' accounts.
But for specifics, we've got to know what you're doing.
EDIT
OK, taking into account what you said below in the comments, you want to prevent unauthorized access to the machine. The second part, ensuring that your users don't do anything bad, is beyond the scope of this particular question, but is a worthy topic too. Broad, but worth asking.
The primary file that you need to edit is /etc/ssh/sshd_config and after each configuration change, you need to run /etc/init.d/sshd restart (or /etc/init.d/ssh if it's a debian/ubuntu system). When you're first learning how to configure ssh, it's a good idea to be logged in on the local console, since a misconfiguration will cut off your access.
Step 1: Make sure that root can't log in via ssh.
PermitRootLogin no
If you absolutely must have root logins, you can set it to "without-password", which requires the connecting user to present a certificate and be authenticated like that.
Step 2: Allow (or deny) any specific users that should (or shouldn't) have access
There are various configuration directives to do this, like DenyUsers, AllowUsers, DenyGroups, and AllowGroups. These take lists of users or groups separated by spaces.
One of the neat things is that you can specify user@host, so if you only want Bob to be able to connect form his home machine, you can say
AllowUsers [email protected]
Step 3: Explicitly permit public key authentication
PubkeyAuthentication yes
This is the default, but we want to ensure it takes effect, because we're going to disable the ability for users to type passwords in...
Step 4: Disable password authentication
Passwords can be stolen, overheard, or copied from the sticky note under your keyboard. Certificates are harder to do. Make sure that people can't use passwords thusly:
PasswordAuthentication no
Step 5: Make sure clients use modern protocols
Protocol 2
OpenSSH supports 2 protocols, called creatively, "1" and "2". "1" is old, allows things like DES encryption and other insecure things.
You're going to have to get your users putty certificates in order to connect now. The easiest way is to use PuttyGEN, which is available at the Putty site (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html).
Hope it helps!