I have a now non-bootable install of Ubuntu. What files do I need to copy to backup all of the MySQL databases (including database users and permissions)?
Since you don't have any dumps, you want the files in
/var/lib/mysql/<dbname>. Since the system won't boot, I would boot with a live cd a mount the hard drive read only.
It should be okay if the DB was not running when when it shut down. If it was running, there might be some corruption. There might also be some file system corruption since it won't boot. Here is a post on recovering databases when they are in a unclean state.
In the future, the proper way to backup databases (if you don't use special software for it) is to create a dump of the database, and then back up those dumps.
The database dumps.
You might be able to copy the data directory to another machine and stand up mysql there. But you will have better luck with actual dumps.