There are several companies providing software for simulating Denial of Service (DoS) attacks. I am curious as to how realistic these simulations can be.
What are the key differences between real vs simulated attacks?
There isn't one - except that if you're paying for it or doing it yourself, you can make it stop immediately when your systems start to crumple.
First and foremost, when simulating you know what the attack is. With a real attack, you may have to do some sleuthing to figure out what is being done. This is pretty easy with network attacks but when they start targeting applications, the diagnostics can be more challenging.
Highly Distributed Attacks are Hard to Simulate
With DDoS, the distribution of the IPs may be far broader than you can easily simulate.
This can play a significant role if you use any type of IP rate limiting tools or behavior-based detection methods.
In one case, I had an HTTP application attack that was flooding a server. We used a counter-measure to drop requests based on request rate of the IP. This worked find until the attackers broadened the attack. They started hitting with thousands of IPs at a rate below our thresholds.
Simulating this type of attack is doable but challenging. In the end, I think we had about 25,000 IP addresses attacking with a near 600Mbs flood. This was all HTTP traffic.