All that happens is that you reduce the file's link count by one. If the link count reaches zero, the space on the disk is marked as available for re-use, but it is not overwritten immediately.
This can bite you two ways:
1) If the file's link count wasn't 1 to begin with, the delete operation isn't actually a delete, it's just an unlink. So the file's contents are still completely available through another path. (Assuming the filesystem is a reasonably modern one such as NTFS.)
2) Even if the file's link count goes to zero, the portion of the disk that held the file data still holds the file data until the system has some reason to write something else there.
Securely shredding a file overwrites the data with random data, zeroes, or the like. This way, even if the data in the file is still accessible somehow, it no longer is data you care about.
Note that for SSDs and file systems with advanced features like data logging and compression, even shredding may not be enough.