For me, the single biggest argument in favour of tape is that doubling your storage capacity is cheap. That is, to go from 1TB of HDD storage to 2TB is the same as going from nothing to that first TB. With tape, you pay a large premium for the drive, but storage after that is comparitively cheap. You don't have to have lengthy budget meetings about increasing the size of the storage NAS by 15TB, you just order another box of LTO5s.
(Chopper makes a valid point about compulsory labels, but tape labels are in a standard format, and there are free software solutions to printing your own onto label stock.)
Tapes are much easier to ship, and easier to store, than HDD and HDD-like media. They're more resistant to shocks, and their temperature tolerances are higher.
They also benefit from the existence of autoloaders. This allows you to spread a large dump over multiple storage containers, which means you don't have to worry about how to break up your backups. While it's perfectly possible to make an autoloader for HDD-type media, I've never seen one, and I suspect the lack of standardisation in physical package size will make it difficult to bring one to market at a reasonable price.
Your point about transfer rates is valid, but in the context of backups it's of minimal import. The time required to back up a 1TB file system to anything is large enough that you shouldn't be doing it on a live file system; and if you're dumping a snapshot to tape, who cares if it takes an extra hour or two? Search times are an equally minor concern, because all decent backup software maintains indices, so one can generally go straight to the relevant portion of the tape to restore a file.