robots.txt doesn't block anything, it is up to the crawler whether it pays attention to robots.txt or ignores it. There's also no central list of web crawlers, since anyone can run one for any reason and they can appear as ordinary browsing traffic, claiming to come from an ordinary web browser.
You can do basic referrer checks to block image hotlinking, you can do intrusion prevention to block port scanners and malicious requests, but if want to block spiders and not people and not false alarms, you probably need to put your site behind a login page.
bots still will "access" and consume the resources.
Minimal resources. You'll spend hours implementing, testing and fiddling with a "spider blocking" policy, and you could spend that investment on hardware that can cope with it instead. It ought to be background noise, really.
If spiders are hammering your site, how will it cope with actual users?