0

I have a massive amount of folders, and I want to zip/gzip/(what ever) them into 3 or more independent files.

So for example, let's say I have this directories on my server:

/dir1
/dir2
/dir3
/dir4
/dir5
/dir6

I need to somehow zip them in 3 separate zip files. That will look like this:

collection1.zip

/dir1
/dir2
/dir3

collection2.zip

/dir4
/dir5
/dir6

collection3.zip

/dir7
/dir8
/dir9

closed as off-topic by dawud, Ward - Reinstate Monica, Evan Anderson, MadHatter supports Monica, Jenny D says Reinstate Monica Jul 10 '14 at 5:24

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions must be relevant to professional system administration. Server Fault is dedicated to professional system and network administrators. End user and enthusiast questions are off-topic (contact your system administrator or hire a professional to help you out). Please see the Help Center for more information." – Ward - Reinstate Monica, Jenny D says Reinstate Monica
If this question can be reworded to fit the rules in the help center, please edit the question.

  • You can use rsync over ssh, no need to zip, no need to delete anything. – dawud Jul 9 '14 at 10:38
  • Target server support only FTP access, no SFTP and my server doesn't have FTP command. So can you maybe give me example of how I would transfer public_html to another server by using rsync? – Jinx Jul 9 '14 at 10:48
  • @dawud i have tried rsync but it uses sftp and target server doesn't support sftp nor shell access. – Jinx Jul 9 '14 at 10:55
  • Maybe you can use ncftp or lftp. Take a look at this Q/A. – dawud Jul 9 '14 at 10:58
  • nope, I don't have those commands – Jinx Jul 9 '14 at 10:59
2

Sorry I dont have a new enough cred to just comment. But something like this should help you with the zipping of the folders.

#! /bin/bash
DIR=$1
i=0
arr=()
zipnum=0
cd $DIR
for item in *
do
    if [[ -d $item ]]; then
            if [ $i -lt 2 ]; then 
                    arr+=($item)
                    ((i++))
            else
                    arr+=($item)
                    zip ${arr[@]} $zipnum
                    ((zipnum++))
                    i=0
                    arr=()
            fi
    fi
done
((zipnum++))
zip ${arr[@]} $zipnum

It should work (didn't have time to test it) Save the file as zip.sh and run is as ./zip.sh /your/directory/here As for the file transfer couldn't you just install an ftp client on the server and ftp the files?

  • Didn't test it either, but I have found my own solution. Thanks for the effort anyway. – Jinx Jul 9 '14 at 16:30
  • This created a zip of folders for me, no files. – Lars Jul 9 '14 at 17:32
  • @Lars judging by his post he only wanted to zip up the folders. If you want files as well just remove the if [[ -d $item ]] statement. – joeg1ff Jul 9 '14 at 20:17
1

The below script was tested on CentOS 6.5.

Walks through the files under a specified folder and adds them one by one to zip file(s) no bigger than a specified max-size. Adding files to a zip one at a time can cause CPU or Storage I/O spikes:

#!/bin/bash

files_folder="/home/username/public_html/"
zip_folder="/home/username/"
zip_maxsize=524288000  # 500 MB

zip_prefix="archive"
zip_suffix=0

cd ${files_folder}

for FILE in $(find . -type f); do
    zip_file="${zip_folder}${zip_prefix}-${zip_suffix}.zip"
    if [ -f ${zip_file} ]; then
        zip_size=$(ls -l ${zip_file} |cut -d' ' -f5)
    fi
    file_size=$(ls -l ${FILE} |cut -d' ' -f5)
    comb_size=$((zip_size + file_size))
    if [ ${comb_size} -gt ${zip_maxsize} ]; then
        zip_suffix=$((zip_suffix + 1))
        zip_file="${zip_folder}${zip_prefix}-${zip_suffix}.zip"
    fi
    zip -q ${zip_file} ${FILE}
done
0

I have found a simple solution.

First I made a list of all folders I need to zip:

ls > list.txt

Than I removed ./ and ../ from the list.

After that I have split that list with text editor into 5 different files, e.g. listPart1, listPart2, listPart3 ...

I put those lists in the folder that contains all folders that need to be zipped and I ran this command from that folder for each list:

cat listPart1 | zip -r@ part1.zip

Not the answer you're looking for? Browse other questions tagged or ask your own question.