I've set up a user account on my Ubuntu server. The user's default shell is rbash. This works great until they type
bash at which point they're not restricted to their home directories any longer. How can I remedy this?
You need to restrict the binaries available to the user, rbash is just the first step.
Here is a blog with further details: http://blog.bodhizazen.net/linux/how-to-restrict-access-with-rbash/
For example, you need to change the user's PATH, to exclude the normal directories like /bin, /sbin, /usr/bin, /usr/sbin. Add a new bin directory to their PATH, and copy place safe commands into it.
As well you need to restrict the user's start up files (like .bashrc) to prevent them from changing their path that way.
You could also do a chroot for more security.
But this is not good security as you are in the position of "enumerating the bad" in step 5 which is essentially impossible as there are many many other ways a user could break out of rbash into another shell.
A better alternative would be to try chroot jails for your users. There are a fair number of relevant tutorials out there, such as this and this as two examples (they are the top two search results for "user chroot", you might want to look deeper or perhaps add a few more search terms to see if you can find information more specific to your environment). Wikipedia has a page on the subject: the limitations section might be of particular interest to make sure the technique can do what you are looking for.