You can not disable individual handler mappings in the UI. The 'Edit Feature Permissions' mentioned by Mark Henderson applies to the whole feature 'Handler Mappings', so it applies to all mappings, not a single one.
There are really three groups of handlers, one that requires Execute permission such as, 'ISAPI-dll' or 'CGI-exe', the second group that requires 'Script' permissions, all the asp.net handlers are in that group. The third group of handlers only requires 'Read' permission, 'StaticFile' is an example of this. Because it does not execute a process nor does it run a script, it just reads a file from the file system.
You can check this by open 'Edit Feature Permissions' and uncheck 'Script', most of the mappings are now disabled. Uncheck 'Read' and the last few enabled ones are disabled as well.
To remove a handler from a site, open the web.config and add something like this:
This will remove the integrate ASP.NET 4 page handler, which means web forms (aspx) will no longer work.
If you look at the 'Handler Mappings' for the same site in IIS Manager, that mapping still shows up in the enabled section, even though it does no longer work for the site.