I have a need to throttle the bandwidth usage, similar to how some ISPs do it, so that after a couple of seconds, it throttles down the speed.
What you're looking for is called Traffic Shaping. The best source of information for it (that I know of) is Linux Advanced Routing and Traffic Control. It's a little bit dated, but still has a ton of good information. If you google for
Linux Traffic Shaping, you can find some scripts and tools to make the traffic shaping configuration a little bit easier.
The specific commands you will use, unless you find a tool or script to do the work for you, are tc and/or tcng. I also want to warn you that traffic shaping can get very complicated if you haven't done it before, particularly if you're trying to do it by hand.
Additionally, also remember that you can only reliably control the speed of data that you are sending out.
You might also want to check out these similar serverfault questions:
We use tc.bash (link) for traffic control, in combination with IPTables. Its done the trick for everything we've needed it for so far.
(Tested on CentOS 4 and 5, and should work with any flavor of linux with tc and iptables.)
For Ubuntu, try wondershaper.