If you're using the ext3 filesystem, consider switching to ext4.
Ext3 can be slow at deleting large files because it stores the location of every individual 4k block: a 50GiB file (50*1024^3 bytes) occupies 13107200 blocks, each of which is recorded in the inode table as a 32-bit block number, for a total of 50MiB of bookkeeping data just to keep track of where the file's contents are located on the disk. That large block list may be scattered across many indirect blocks, all of which have to be updated when the file is deleted. Disk seeking to access all those indirect blocks is probably what's causing the delay.
Ext4, on the other hand, allocates files in "extents" of up to 128MiB. That 50GiB file can be recorded in the inode table using just 400 extent records, rather than 13107200 individual block numbers, which dramatically reduces the amount of disk I/O needed when deleting the file.
Note that if you convert an existing ext3 filesystem in-place into ext4, new files will be allocated using extents, but existing files will still use block lists. You can use the
chattr +e command to reallocate an existing file using extents; performance-wise, this is comparable to making a copy of the file and then deleting the original.