If you would like a tool that now comes packaged with MySQL 5.1, you could do worse than try MySQL Slap (aka mysqlslap). An example straight from the documentation:
Supply your own create and query SQL
statements, with 50 clients querying
and 200 selects for each:
mysqlslap --delimiter=";" \
--create="CREATE TABLE a (b int);INSERT INTO a VALUES (23)" \
--query="SELECT * FROM a" --concurrency=50 --iterations=200
This supplies the option to run the tests in parallel - although the implementation of parallelism using multiple clients may not be exactly to your liking. Here is a brief description (again, verbatim):
mysqlslap runs in three stages:
Create schema, table, and optionally
any stored programs or data you want
to using for the test. This stage uses
a single client connection.
Run the load test. This stage can use
many client connections.
Clean up (disconnect, drop table if
specified). This stage uses a single
If this is not exactly to your taste, you could either use a cron script to trigger MySQL Slap at a specific time for many users, or for many identically configured/spec'd machines, with identical network paths to server etc (the latter is very important, as benchmarking should eliminate any possible discrepancies to prevent obscured conclusions).
If (as is likely) you are running a previous MySQL version, this link provides useful information on how to compile the 5.1 source for 5.0; further backporting is probably not possible.
Here is a basic walkthrough.