0

We will be upgrading our RHEL 6.5 machines to RHEL 6.6 in the coming week. I know how to create repos using Puppet, but instead of just using exec to run a yum -y update, is there a way to tell Puppet to bring (or keep) the OS to a certain release? Something like stating that it should change the sytem to reach or keep the following criteria:

operatingsystem => RedHat
operatingsystemmajrelease => 6
operatingsystemrelease => 6.6
3
  • Downvoter, could you add a comment and say why?
    – Sreeraj
    Dec 13, 2014 at 13:56
  • 1
    I assume it is because it's somewhat hard to understand what you want to do. I think I've an idea about it, but I am not sure. If you don' agree with my edit, just roll it back.
    – Sven
    Dec 13, 2014 at 14:14
  • mcollective is better for this kind of scenario.
    – jordanm
    Dec 27, 2014 at 0:19

2 Answers 2

1

TL;DR this is not a good idea. Do the update manually, or with a different form of automation.

Why it's a bad idea

I don't think that you should go down this path at all.

Generally, the idea to use Puppet to make sure that an update of the OS has been successfully completed is sound, at least from an academic standpoint. Puppet's goal is to allow you to define a state, and take care of the specifics of reaching this state.

That being said, Puppet would require a (fictional) distro type, so that you could specify

# pseudo code! Do not try at home or at all!
distro {
    'CentOS',
        version => '6.6';
}

You could conceivably go ahead and implement such a type, with the capability to try and perform a synchronization action to get from whatever your current release is to the one you have requested.

However, such a process can be infinitely complex and has virtually unlimited possibilities for failure and system corruption (of done wrong or at all). As such, it does not really lend itself very well to any form of automation.

As for Puppet specifically, you would want to add logic to your distro type/provider to recover from all sorts of weird states and achieve a clean one. The mere thought of such an endeavor is making me dizzy.

What would be less painful

Write a shell script. If the number of machines is too large to do them in batches using cssh (as I would probably do for anything below a few hundred nodes), create a simple wrapper that will do all you need. Deploy it with Puppet, and yes, do use an exec resource if you want Puppet to trigger the update. Consider

exec { 'echo /my/update/script | at now+10min': }

so that the puppet agent process is not the parent of the yum instance that will do all the work. This might prove disastrous.

Still, I'd launch the yum commands manually if it is at all feasible.

3
  • Thanks. How about using the exec resource to run the yum -y update ?
    – Sreeraj
    Dec 15, 2014 at 16:56
  • I'd be scared of doing that. Imagine the yum update process crashing and burning due to some timeout from Puppet, or a weird interaction, or anything else you haven't thought about. Imagine this occurring on all of your machines, in a course of about 30 minutes. Now you are salvaging a whole network from a botched update, taking 4x the time a manual approach would have taken you. - That being said, if your tests indicate that this will Just Work, sure go ahead :-) [Full disclosure, I don't manage RHEL based systems a lot and have updated few.] Dec 15, 2014 at 16:59
  • Once the yum -y update is fired, I don't think it is going to crash because of any kind of time-out. The yum is not going to run as a thread of the puppet agent, but as an independent process. It should be okay.
    – Sreeraj
    Dec 15, 2014 at 17:22
1

I agree that upgrading an OS through puppet looks like being too error-prone and at last unpredictable. But this isn't really true if you follow three rules:

  1. Manage machines with a maximum level of standardization regarding OS configuration. To get this you have to manage them strictly through puppet.
    No pets, only cattle!
  2. You have at least one test stage.
  3. Your deployment mechanism assures that the exact puppet code you tested runs on the next stage.
  4. The upgrade itself has to be the very last step on a puppet run.
  5. As already discussed, the command for a necessary reboot must not be a parent of the puppet process. Otherwise the puppet run ends with an error at least.

I use the following puppet code for upgrading ubuntu machines from 12 to 14:

class os_upgrade {  
   # Script for automatic reboot after the puppet run
   file {'/tmp/reboot_after_puppetrun.sh' :
     content => "watch -g ls -l ${::puppet_vardir}/state;nohup /sbin/reboot\n",
     mode    => '0755',
   }  
   # Unattended upgrade
   exec { 'do-release-upgrade' :
     command   => '/usr/bin/do-release-upgrade -f DistUpgradeViewNonInteractive -m server',
     logoutput => true,
     onlyif    => "test $::operatingsystemrelease != '14.04'",
     notify    => Exec['reboot_after_upgrade'],
     timeout   => 1800,
   }  
   # Trigger the reboot as a background job to avoid puppet parentship
   exec { 'reboot_after_upgrade' :
     command     => "/usr/bin/nohup /tmp/reboot_after_puppetrun.sh&",
     refreshonly => true,
   }
}

To meet point 4 you have to define a stage and make sure the above mentioned class is attributed correctly.

In site.pp:

stage {'post': subscribe => Stage['main']}
node 'your_machine' {
  class { 'os_upgrade' :
    stage => 'post',
  }

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .