Title says it all... does it support multiple databases, or do I have to create a new instance for each database?
Yes, you can create multiple databases per instance, see their respective FAQ: What is a database instance (DB Instance)?
You can think of a DB Instance as a MySQL 5.1 environment in the cloud with the compute and storage resources you specify. You can create and delete DB Instances, define/refine infrastructure attributes of your DB Instance(s), and control access and security via the AWS Management Console, Amazon RDS APIs, and Command Line Tools. Multiple MySQL databases can be created on a given DB Instance.
You can create databases using normal SQL commands, eg:
CREATE DATABASE <database_name>;