3

I have a bunch of servers that have two directories on them that need to be in sync. For instance, on the same server I might have /var/path/to/dir1 and /var/path/to/dir2 which could have files and folders underneath of each. I'd like to sync /var/path/to/dir1 and have /var/path/to/dir2 as the "backup".

Does anyone know an efficient way in Puppet to sync these two directories? Would I have to create a custom script to do this and use the exec method in the Puppet manifest file?

My guess was to create a script to recursively verify the MD5 hash on every file in the primary directory and then verify the files in the second directory with that list. I wasn't sure if there was a way to do this in Puppet natively.

3
  • No option to rsync?
    – ewwhite
    Nov 28, 2014 at 22:22
  • 1
    There is actually a rsync module: forge.puppetlabs.com/puppetlabs/rsync
    – faker
    Nov 28, 2014 at 22:23
  • Yeah I thought about that, but it's only local on each server since these config files are different on each box. I don't have a central repo. Also, I might have Windows machines running this app... Can the File resource work with the recurse and purge attributes set to true?
    – Jeff
    Nov 28, 2014 at 23:28

2 Answers 2

3

You were on the right track with puppet. Have the puppetmaster serve as the source of truth, house them in the server that runs your puppetmaster, and declare the files that should exist and the canonical versions there. You may even want to version the files that you are making available in the puppetmaster server using something like SVN or GIT.

This provides you some strong guarantees about what is in those two locations.

file { '/var/path/to/dir1/':
  ensure   => 'present',
  source   => 'puppet:///path/to/dir1',
  recurse  => 'true',
} 
file { '/var/path/to/dir2/':
  ensure   => 'present',
  source   => 'puppet:///path/to/dir1',
  recurse  => 'true',
}

As mentioned above, you may need to remove any collected unwelcome extras with something like

{
  purge => 'true',
  force => 'true',
}

As was pointed out to me (thanks!!), if there is a large number of files or a deep hierarchy of sub-folders under '/var/path/to/dir1/', the time to complete a puppet run could grow rapidly. I would still recommend to keep your puppet server as the source of truth, but you might do better to use rsync, something like:

rsync::get { '/var/path/to/dir1/':
  source  => 'puppet:///path/to/dir1',
  purge   => true,
}

rsync::get { '/var/path/to/dir2/':
  source  => 'puppet:///path/to/dir1',
  purge   => true,
}
4
  • Beware that syncing deep directories with many files like this will make the puppet run take ages, because puppet doesn't run chmod -R. Using rsync (see @faker's comment) should accomplish the same and make it run faster.
    – fuero
    Nov 29, 2014 at 12:11
  • @fuero Good point. I had forgotten about that. I will update my answer.
    – DTK
    Nov 29, 2014 at 12:13
  • Thanks everyone for your help - it looks like this works perfectly. Good to know about large directories, too.
    – Jeff
    Nov 29, 2014 at 15:40
  • @FelixFrank Thanks for the help making this look more professional and for catching the slash-miscount.
    – DTK
    Nov 30, 2014 at 11:33
0

I got curious and tried it out:

[root@localhost vagrant]# cat test.pp
file { '/tmp/path2':
  ensure  => directory,
  source  => 'file:///tmp/path1',
  recurse => true,
  purge   => true,
  force   => true,
}

[root@localhost vagrant]# ls -lR /tmp/path1
/tmp/path1:
total 4
-rw-r--r-- 1 root root    0 Nov 29 01:55 bar
drwxr-xr-x 2 root root 4096 Nov 29 02:07 folder
-rw-r--r-- 1 root root    0 Nov 29 01:55 foo
-rw-r--r-- 1 root root    0 Nov 29 01:55 x

/tmp/path1/folder:
total 0
-rw-r--r-- 1 root root 0 Nov 29 02:07 xxx
[root@localhost vagrant]# ls -lR /tmp/path2/
/tmp/path2/:
total 4
-rw-r--r-- 1 root root    0 Nov 29 02:17 deleteme
drwxr-xr-x 2 root root 4096 Nov 29 02:17 folder2

/tmp/path2/folder2:
total 0
-rw-r--r-- 1 root root 0 Nov 29 02:17 deleteme
[root@localhost vagrant]# puppet apply test.pp 
Notice: Compiled catalog for localhost in environment production in 0.04 seconds
Notice: /Stage[main]/Main/File[/tmp/path2/x]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: /Stage[main]/Main/File[/tmp/path2/deleteme]/ensure: removed
Notice: /Stage[main]/Main/File[/tmp/path2/folder]/ensure: created
Notice: /Stage[main]/Main/File[/tmp/path2/folder/xxx]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: /Stage[main]/Main/File[/tmp/path2/bar]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: /Stage[main]/Main/File[/tmp/path2/folder2]/ensure: removed
Notice: /Stage[main]/Main/File[/tmp/path2/foo]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: Finished catalog run in 0.06 seconds
[root@localhost vagrant]# diff -r /tmp/path1 /tmp/path2
[root@localhost vagrant]#

So that works.
Note that you need force => true, too.
Otherwise existing folders in the destination path will not get removed.

Or something like this will also work:

rsync::get { '/backup/foo':
  source  => '/foo',
  purge   => true,
}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .