You certainly would see some benefit from using a CDN I'm sure, but it does depend on what the cause of speed problems are on the server. Anything you offset from your server would speed things up, particularly give the geographical nature of a CDN.
Are people suffering slow speeds due to latency or because your bandwidth is routinely getting saturated, or is it that your server is working so hard it can't keep up with all the requests? Or is it something else entirely?
All 3 would show some benefit from the use of a CDN, but whether its the most appropriate solution is debatable. The CDN would reduce load on your server hosting static content, and if you're saturating your connection will ensure everyone gets fast downloads; but if your page takes 8 seconds to generate it won't be of any huge gain (I know that's not really what you're saying with that 8 seconds, just trying to help clearly identify the benefits)
From a strictly web server perspective, if you're running on a *nix server have you considered using nginx or lighttpd instead of Apache or as a reverse proxy to it?
Both are extremely fast at serving static content like images and use significantly less memory than Apache. I'm not sure about lighttpd, but I do know nginx can be told to serve from a pre-compressed source location for example, giving you the benefits of gzip'd files like reduced bandwidth usage without the CPU hit of on-the-fly compression.
Also Varnish as a caching reverse-proxy might be worth considering instead. If your server is reasonably busy even caching a page for 30 seconds - 1 minute can see huge benefit.
A large majority of sites if you get down to the fine detail of it don't actually need to be completely dynamic.
The company I work for uses Drupal as a CMS for the website, but recently settled on generating the content as static pages through a simple script that runs every 10-15 minutes. The servers providing it have now gone from dynamic to almost entirely plain static hosting and absolutely blaze along. Even re-generating pages that frequently is a bit overkill for our purposes. The only section that needs to be dynamic, is!