Optimal way to build and deploy a website as a .deb package

Question

I've been battling with deploying (mainly PHP) websites as .deb files for some time and wonder if there's a better way than my rather convoluted method. My goal is continuous integration on my staging server and one-click deploys to the live server from within Jenkins.

My use case now:

• Developing code on OSX
• Version control in Git
• Source contains debian/ folder with control, postinst and prerm scripts
• Phing build script run locally in a fakeroot environment
  • copies files into a /tmp build folder mirroring the filesystem layout
  • sets up file ownership as root / www-data as appropriate
  • runs dpkg-deb --build ${build.dir} ${working.dir}
  • copies package to private deb repository on my LAN

I have tried to set up a Jenkins CI server to watch the Git repo for commits and automatically perform a build. The problem is that Jenkins runs as its own user who does not have permission to set the file permissions as above. I also can't see an obvious way to run the Phing build in a fakeroot environment from within Jenkins.

My questions are:

• Is there a better way to set up the correct file ownership and permissions rather than having to copy everything to a temporary dir and have a Phing step to chmod everything?
  • Does the dpkg-deb command really need all the file permissions set up on the filesystem beforehand? Is there something that I could put in the debian/ folder that would set up these permissions when the .deb is installed?
• How can I get Jenkins to run a build script that has the correct rights to set up file ownership?

I may have misunderstood how .debs work but it seems rather inconvenient that the file ownership on your local filesystem source code has to mirror the destination!

I'm open to suggestions on any other build and CI systems that people use to perform a similar task.

Answer 1

It seems that you create a binary debian package almost by hand using dpkg-deb command. While this approach is not that bad, you'll get better handling of a lot of things if you try to build packages by creating the source packages and then building binary packages out of those (even if it's architecture-independent files like PHP ones). This requires one-time configuration of several files in debian/ directory.

There is a special debian/rules file which decides what to run in order to build Debian packages of different flavors (i.e. source, binary, binary-independent) and to maintain build directory itself. You might consider running all your build tools from this file rather then invoking them one by one. To fix the problem with setting owners/permissions during the build from the unprivileged user you would need to run the whole process of building Debian packages under fakeroot wrapper. For official debian packages this can be achieved by running fakeroot debian/rules binary. However this is invoked from the dpkg-buildpackage so you just run fakeroot dpkg-buildpackage.

Debian is great by its developer tools (see packages debhelper, devscripts and related) and it appears you're not using them too(obviously because you build the packages under another operating system). They will save you a lot of time. For example, there are dh_fixperms which will fix permissions and ownership problems for you, dh_install to place files into the correct places, dh_link to create necessary symlinks and other dh_-like scripts. You might want to see how does it work in the real world, so here is a list of software implemented in PHP and packaged in Debian.

Another good tool for you will be git-buildpackage which is a wrapper for debian build tools designed to build packages maintained under Git DVCS.

You might want to look at general information for building with Git on Debian wiki page.

Unfortunately there is no phing in Debian, so you might want to ask someone to package it for Debian (so you would have your full build stack available in Debian) or just cook a Debian chroot with debootstrap and install phing manually in it.

If you're sticking to the Debian packages as a primary deployment mechanism together with some CI system here are some points:

1. Get the whole build stack into Debian, if possible. This will give you a possibility to build your software in a clean Debian chroot using tools like pbuilder or cowbuilder. Otherwise cook a Debian box for building. You might want to create a virtual machine and give it for other developers.
2. Organize a restricted Debian repository (creating GPG key for automatic signing for the repository is sufficient to reduce warnings coming from apt) and put the packages there. Add it to the target boxes' /etc/apt/sources.list.d/my-repository.list configuration file. Don't forget to import your key. This will give you information on dependencies you wouldn't have known if you installed the package with just dpkg -i *.deb until the installation phase itself.
3. You might want to install your sites from packages on some regular basis (like releases/nightly/etc) instead of installing them every build to save time and bandwith. I'm using mostly rsync and some Makefile to update database(idea borrowed from ruby database schema migration).
4. You might want to run some administrative commands from CI agent without password(for example, reloading webserver configuration when it's updated with sudo invoke-rc.d nginx reload. If so, restrict to run exact commands by modifying /etc/sudoers file respectively.

Answer 2

A Debian package is a multi-part archive that contains the data, and control information /debian. The data archive is basically extracted as-is. The control archive is extracted and mostly moved into /var/lib/dpkg/info and the pre|post-inst|rm scripts are called when appropriate. If you want to change something after your files are extracted then do it in your postinst.

but it seems rather inconvenient that the file ownership on your local filesystem source code has to mirror the destination!

Generally people will build a chroot or virtual machine that mirrors the architecture they are building the package for. Trying to build a package for an Debian system from OSX is pretty uncommon.

Does the dpkg-deb command really need all the file permissions set up on the filesystem beforehand?

That is the normal way permissions are handled. Modifying permissions postinstall is the exception and not the rule.