Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I copied this script from some book to make tar.bz2 of some folders for backup.

#!/usr/bin/env python
import tarfile, os

def make_tar(folder_to_backup, dest_folder, compression='bz2'):
        if compression:
                dest_ext ='.' + compression
        else:
                dest_ext = ''
        arcname = os.path.basename(folder_to_backup)
        dest_name = '%s.tar%s' % (arcname, dest_ext)
        dest_path = os.path.join(dest_folder, dest_name)
        if compression:
                dest_cmp = ':' + compression
        else:
                dest_cmp = ''

        out = tarfile.TarFile.open(dest_path, 'w' +dest_cmp)
        out.add(folder_to_backup, arcname)
        out.close()
        return dest_path

print "Doing Python"
make_tar('/home/bob/public_html','/home/bob/testbck', compression='bz2')

Now bash take 40 second to make backup of that folder and python takes around 8 minutes.

Am i wrong somewhere or python is always slower for these tasks

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

I copied/pasted your code and tried with both bz2 and gz against tar cjpf and tar czpf respectively and found them to perform the same. Which version of Python are you using? How many files are there on /home/bob/public_html? Did you try the tar command first and then your script or the other way around? (I'm guessing file caches may skew the results a bit, but not so much though).

I just took a look to TarFile's implementation. It's easy with ipython, by the way:

import tarfile
%edit tarfile.TarFile.add

And this is the case for directories:

    elif tarinfo.isdir():
        self.addfile(tarinfo)
        if recursive:
            for f in os.listdir(name):
                self.add(os.path.join(name, f), os.path.join(arcname, f), recursive, exclude)

Which I can see getting slower as the number of total files increases. I'm guessing tar may be more optimized when handling this case. It's just a guess, though.

share|improve this answer
    
its working now , actually i was compairing linux tar.gz with pyhton tar.bz2 which had different times , now i have chnaged to gz and its almost same. that was my first python script . very happy now –  Mirror51 May 16 '11 at 12:37
    
Ah, that explains it of course! I should have thought of that possibility. Have fun with Python and take a look at ipython, it lets you try your code on the go much more easily than the standard interpreter. You can also call it from your scripts to "debug" interactively. –  Eduardo Ivanec May 16 '11 at 12:43
    
i am using linux termnal by writing .py files and then executing. how can ipython help me . does it has more commands and also how will my .py script chnage if i want to use ipython –  Mirror51 May 16 '11 at 16:53
    
No new commands or anything - it's just a way to execute python interactively, allowing greater flexibility when developing. No need to use it for your script at all, it was more of a general recommendation. –  Eduardo Ivanec May 16 '11 at 17:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.