Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a CentOS server on a production environment. I need to update the PHP package that I installed using the REMI repository.

Quite easy:

yum update php

But what is it going to happen if something goes wrong during the update? How can I rollback?

What's the best technique to make sure not to compromise a production server due to an update?

Is it maybe better to compile PHP from the source, rather than using a binary package?

EDIT: I am not afraid of incompatibility between my code and the new PHP version (I have well tested that on development). I am more afraid of something going wrong while CentOS updated the binary (power cut, lost connection, unexpected conflit)

Thanks,
Dan

share|improve this question

3 Answers 3

The binary package has been built by your vendor and has generally had a fair amount of testing. By building from source, you lose most of the advtantages of your packaging system, specifically:

  • The packaging system lets you verify the integrity of installed packages using checksum.
  • The packaging system is aware of dependencies between packages and can automatically install the necessary additional software to make things work.
  • The packaging system lets you conveniently remove a package, whereas most source installs don't really provide a good uninstall mechanism.
  • Using packages means it's very easy to install an older version of the package of there are problems with the new version.

There are certainly some advantages to building from source -- for example, you may require a different configuration than provided in the vendor package. However, even if you elect to compile the code locally, it's almost always a better idea to create a package from it and install it using the normal packaging tools for above reasons (rather than running make install directly into your filesystem).

What's the best technique to make sure not to compromise a production server due to an update?

Test the updates in your development environment to make sure they work as expected before deploying them on your production system. Ensure that your development environment accurately reflects your production environment.

This is true regardless of how you choose to install the updated software.

share|improve this answer
    
Thanks. I have tested my code against the new PHP version. It works fine. What I am afraid of is, while updating the PHP package on the production server, something goes wrong (lost of connection, power cut, unexpected conflict, human error). What I am afraid of is to end up with a new version installed half-way with no possibility to rollback. –  dan Jan 13 '11 at 1:32
1  
There's always the possibility of rollback -- you can install the previous version of the package, or re-run the upgrade. If you use a tool like screen, you don't need to worry about loss of connection, etc. Conflicts will simply cause the whole upgrade to abort without making any changes on your system. Power loss and human error can always cause problems regardless of how you choose to install; this is what UPS's and backups are for. –  larsks Jan 13 '11 at 1:35

The absolute best thing you can do in this case is thoroughly test your set of packages as a release on a test server, including rollback to the previous set or packages. I work in release management in a very large internet company and we do exactly this.

By doing thorough installs and rollbacks before going to production you eliminate nasty surprises (make sure to review your install logs!). You also need to consider doing bare metal restores, where you take a host with a basic OS installed, and install your release on top of that. You will often find many hidden surprises when you install all the packages together simultaneously this way.

Try to take hosts out of rotation if possible when doing production installs, and do an automated healthcheck to determine they are working properly when the install completes. That way if something goes catastrophically wrong on one host, you can leave it our of rotation. Then come back after your software push, wipe the host, and reinstall. Obviously that approach is dependent on having redundant servers set up.

share|improve this answer
    
You are so right. Unfortunately I can't give any point to your answer because I haven't got enough reputation :-( –  dan Jan 13 '11 at 10:22
    
Well dan be sure to come back and give me an upvote after you've got some more reputation. :) Also you should accept one of these answers please if you are satisfied. –  Phil Hollenback Jan 13 '11 at 21:54

You're probably best off using the binary package, assuming the current version was installed as a binary. Do you have any test environment in which to run a trial of this new PHP package? What version of PHP is currently installed? Run rpm -qi php. What is the version that you're looking to install?

share|improve this answer
    
Running 'yum list php', the output is: Installed Packages 5.3.3-1.el5.remi, Available Packages 5.3.5-1.el5.remi –  dan Jan 13 '11 at 1:29
    
I tested my code against the new version of PHP. It works fine. What I am afraid of is, while updating the PHP package on the production server, something goes wrong (lost of connection, power cut, unexpected conflict, human error). What I am afraid of is to end up with a new version installed half-way with no possibility to rollback –  dan Jan 13 '11 at 1:31
    
It will take under 10-15 seconds to actually replace the package. It's also a minor version upgrade, so your configuration files probably won't change. The upgrade will probably restart Apache (httpd) at the end of the upgrade. This is fairly minor, so you should probably just go for it. –  ewwhite Jan 13 '11 at 1:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.