Update: I'm looking for an automated "95th percentile bandwidth limiter". I was originally hoping for something part of the traffic shaping code in the kernel but I think it doesn't exist. I'm aware that I can manually limit at 3Mbps and be safe, but that would be wasting a lot of bandwidth that I could be using.
Let's assume I have a set of co-located servers and I pay for 3Mbps at 95th percentile. I would like to use as much of that as possible without incurring overage charges. Without any throttling, the servers would generate 10Mbps of traffic, leading to overage charges. What I'm asking is for something that would not throttle at all for 5% of the time becuase the 95th percentile billing rules will throw those out, while keeping me under the 3Mbps the rest of the time.