Let's assume I have a several gigabyte tar file, but I also happen to know that the very last file written to the archive is something important that I need. Since tar files are appended sequentially, is there a way I can make tar read into the archive from the end to find this file, instead of starting from the beginning and reading over gigabytes of irrelevant data?
No, unfortunately there is not. From Wikipedia
Another weakness of the tar format compared to other archive formats is that there is no centralized location for the information about the contents of the file (a "table of contents" of sorts). So to list the names of the files that are in the archive, one must read through the entire archive and look for places where files start. Also, to extract one small file from the archive, instead of being able to lookup the offset in a table and go directly to that location, like other archive formats, with tar, one has to read through the entire archive, looking for the place where the desired file starts. For large tar archives, this causes a big performance penalty, making tar archives unsuitable for situations that often require random access of individual files.
We can efficiently seek to last file on archive if tar is created on storage which is seekable, i.e. on hard disks and not on tape. use GNU tar's -n or --seek option. ( see this GNU tar options page ) say for instance, file which is stored last with name last_file.txt , you could use following command
tar -nxvf <your_archive> last_file.txt
Which would simply extract last_file.txt. because tar format contains size of a each file in a header, it is possible to skip entire file efficiently using seek system call, (see tar file format )
To only list all files efficintley in a big archive, use
tar -ntvf <your_archive>