You should avoid trying to do this with PHP. By the time PHP gets involved, it's already too late - the memory has already been allocated.
You can ban IP addresses at any layer, but the lowest level that uses the least amount of resources is the route you want to take. This is usually the firewall. At the very least, iptables (linux firewall) is what you want to use. There are tools that others have mentioned, such as Fail2Ban, that can automate this for you. External firewall would be better.
Besides trying to ban offending IP addresses, you should try to make better use of your resources. If a request takes less resources it will take longer for an attack to be effective.
Apache also uses a lot of memory. If you're using mod_php, it's even worse because PHP is loaded inside of every Apache child process. This means even requests to static content (css/js/images) are loading PHP even when PHP isn't being used. You can solve this problem by using FastCGI instead. mod_fcgid is a good option.
There are also other web servers that are more resource efficient. Nginx is my favorite. There's also Lighttpd. A lot of people like Litespeed (drop in replacement for Apache).
If you want to stick with Apache, consider tuning it as best you can. Consider disabling .htaccess. Here's a good explanation why.