2

I have a script1 which has around 60 lines similar to this

mv /log/ports-01/Homepage.log.$a /log/Backup/ports-01/

Also I have another script2 which has one set of commands like this for all the move commands in the above script1

tar -cvf /log/Backup/ports-01/Homepage_$a.tar /log/Backup/ports-01/Homepage.log.$a
gzip /log/Backup/ports-01/Homepage_$a.tar
rm -f /log/Backup/ports-01/Homepage.log.$a

Script2 runs after 2 hrs from the time script1 runs. There seems to be no problem with the directory structure.

But the problem is that not all the logs are getting moved and tarred. Also many tar files that get created are empty but the logs files get deleted.

Also I want to ask if there is any problem in having so many lines in one script.

1
  • Sounds like you may have a full fielsystem. What does your output from "df" look like. As you are "tar"ing to the same directory you will require a least double the space of all your source files.
    – user23523
    Nov 17, 2011 at 7:54

3 Answers 3

2

From what I could deduce, the problem may be the variable 'a'. Try

mv /log/ports-01/Homepage.log."$a" /log/Backup/ports-01/

That is, try quoting the variable reference(make sure you do this with all occurrences of the variable. Also, as a tip, you can use the z option of the tar utility to create tarball directly, like so: tar czvf /path/to/ball.tar.gz /path/to/dir

3
  • Thank u... But if that is the problem it shouldn't work for any of the logs...as i mentioned in the post few logs are getting tarred and compressed properly....
    – Hawks
    Nov 17, 2011 at 8:14
  • 1
    if some of the logs have a name with a space or other special char in it, not quoting the var will break things. Nov 17, 2011 at 8:36
  • When I run those two scripts manually they work fine.. Whereas if I schedule it in a cron job this problem is there..
    – Hawks
    Nov 18, 2011 at 6:14
1

Well if runs fine when you execute it manually then there is probably problem with environment variables, check if your script depends on some environment variables that you don't have in CRON environment.

Also if you execute two script and second depends on first, do you execute them one after another, like:

0 */2 * * *  /home/username/first_script; /home/username/second_script

It is trivial, but just to check.

If you can't find answer that way, you could use "logger" command to store logs of all steps of your scripts, and try to catch your bugs that way.

1

First off, there is no problem having a script longer than 60 lines - I routinely deal with shell scripts that have hundreds of lines in them, the shell doesn't care.

Without more information (show us your crontab entry/entries, and the whole script being run - how is this "$a" populated? Does it contain spaces? etc.) it's hard to give you a good, definitive answer as to why this isn't working, but I can give you some food for thought:

I have an issue with the way you execute your scripts: Using timing to separate dependent actions is a recipe for disaster. Depending on how large these files are, how slow your disks are, whether you're crossing partitions, etc. etc. etc. it's possible your first script isn't finishing when your second script runs. This has two major implications:

  1. Script 1 might still moving files over while Script 2 is making its tarball.
  2. Script 2 is only tar'ing the stuff it sees when it executes the tar command.
    • Script 2 is also rm'ing what it sees when it gets to the rm command -- This could mean it's deleting stuff that wasn't added to the tarball
    • If Script 1 is still running you'll have "stray" files copied over after the rm is done - These will just sit around.

@XoR's suggestion of chaining the scripts in cron is one way to protect yourself. So is combining both scripts into one larger script.
Using lock files (Linux has the lockfile(1) command ; I believe most of the BSDs have lockf(1)) is another option, and is somewhat more robust than chaining scripts together in cron.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .