jenkins-debian-glue

Manual Deployment of jenkins-debian-glue

Table of Contents

System requirements

Preamble

jenkins-debian-glue assumes two (or optionally more) Jenkins jobs for each software package. One job builds the Debian source package, this is the source job. Assuming the software package is named foobar then its source job is assumed to be foobar-source. The other job builds the Debian binary package(s), this is the binaries job. Once again assuming we're talking about the foobar software package then the binaries job is assumed to be foobar-binaries. As we're starting with jenkins-debian-glue itself we'll use jenkins-debian-glue-source and jenkins-debian-glue-binaries in the following documentation.

Setting up those jobs is really simple. Just follow the following steps and you should have a working system in less than 20 minutes.

Install Jenkins Plugins

Install "Copy Artifact Plugin"

The Copy Artifact Plugin (install it via http://$JENKINS_SERVER/pluginManager/available or through the jenkins-cli.jar's install-plugin option on the command line) provides the option to deploy artifacts from the source job to the binaries job. Make sure it's installed so you get the "Copy artifacts from another project" option as Build step, used in the following steps.

Install "Jenkins Workspace Cleanup Plugin"

The Jenkins Workspace Cleanup Plugin provides the "Delete workspace before build starts" option which is useful for getting rid of files from previous runs.

Install and configure VCS tools

Setup for Debian packages managed in Git

Make sure the Git Plugin is installed in Jenkins. On the build system you need git-buildpackage and a basic Git configuration for the user under which Jenkins is running (usually being 'jenkins'): 

# apt-get install git-buildpackage

Then either adjust the "Git plugin" settings at http://$JENKINS_SERVER/configure or manually execute:

# su - jenkins
% git config --global user.email "jenkins@example.org"
% git config --global user.name "Jenkins User"

Setup for Debian packages managed in Subversion

The "Subversion Plugin" is shipped by Jenkins out of the box nowadays, but to use it with Debian packages please make sure you've subversion-tools available on the build system:

# apt-get install subversion-tools

Create the source job configuration (jenkins-debian-glue-source)

Create a new job using the 'Build a free-style software project' option in Jenkins. Use jenkins-debian-glue-source for the job name. Then enable and configure 'URL of repository' of the according version control system under 'Source Code Management' to automatically check out sources.

Important: set the "Check out to a sub-directory" option under "Source Code Management -> Git -> Additional Behaviours" to "source". (For jobs using Subversion set "Local module directory (optional)" to "source").

In the "Build" section add a build step "Execute shell" as follows to get rid of files from previous runs without removing the VCS checkout:

rm -f ./* || true

Notice: the "Delete workspace before build starts" option provided by the Jenkins Workspace Cleanup Plugin works as well, though it also removes the repository checkout so build time might increase. Identify what you want and choose on your own.

Then add another build step of "Execute shell" as follows:

# when using git:
/usr/bin/generate-git-snapshot

# when using subversion:
# /usr/bin/generate-svn-snapshot

Under "Post-build Actions" select "Archive the artifacts" using: 

*.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes,*.buildinfo,lintian.txt

Screenshot of Archive the artifacts

Enable "Record fingerprints of files to track usage" and its "Fingerprint all archived artifacts" setting:

Screenshoft of Fingerprint all archived artifacts

Create the binaries job configuration (jenkins-debian-glue-binaries)

Add a new "Build multi-configuration project" Jenkins job named jenkins-debian-glue-binaries.

Enable "Build after other projects are built" under Build Triggers and choose the name of the source job, being jenkins-debian-glue-source in this example:

Screenshot of Build After Other Projects Configuration

Under Configuration Matrix create a user-defined Axis titled "architecture" specifying the architectures your Debian packages should be built for. If you're running an amd64 system (recommended nowadays) then amd64 should be your default choice no matter what. If you also want to build Debian packages for i386 then add i386 (separated by whitespace from 'amd64'):

Screenshot for user defined axis

Choose "Run each configuration sequentially" and in the Build Environment section enable the "Delete workspace before build starts" option (provided by the Workspace Cleanup Plugin):

Configuration for Build Environment

Notice: once you're building for multiple architectures or distributions you might also want to enable the option 'Execute touchstone builds first'.

In the "Build" section configure the build step "Copy artifacts from another project" as follows:

Screenshot of Copy Artifcats configuration

In the "Build" section add a build step "Execute shell" using: 

/usr/bin/build-and-provide-package

Under "Post-build Actions" select "Archive the artifacts" using:

*.gz,*.bz2,*.xz,*.deb,*.dsc,*.changes,*.buildinfo

Configure sudo

To be able to execute cowbuilder as user jenkins install a file into /etc/sudoers.d/jenkins (file permissions 440 recommended) containing:

jenkins ALL=NOPASSWD: /usr/sbin/cowbuilder, /usr/sbin/chroot
Defaults env_keep+="DEB_* DIST ARCH"

The second line makes sure that DEB_* and DIST environment variables are propagated to cowbuilder.

Create /srv/repository/ and provide write permissions to jenkins user:

# mkdir /srv/repository
# chown jenkins /srv/repository

Enable Lintian reports

Lintian dissects Debian packages and tries to find bugs and policy violations. It contains automated checks for many aspects of Debian policy as well as some checks for common errors which turns out to be very useful for inclusion in Continuous Integration/Delivery infrastructures.

To enable Lintian reports for your jenkins-debian-glue jobs install the jenkins-debian-glue-buildenv-lintian package, add the following "Execute shell" build step at the end of the build steps in your *-source and *-binaries Jenkins jobs:

mkdir -p report
# For the source package:
/usr/bin/lintian-junit-report *.dsc > report/lintian.xml
# or, for the binary package:
/usr/bin/lintian-junit-report *.changes > report/lintian.xml

and add lintian.txt (no it's not a typo) to the list of artifacts to archive.

Enable the 'Publish JUnit test result report' Post-build action and select **/lintian.xml for the files to report.Then you should get test reports for your Debian packages based on lintian's output.

Building the Jenkins jobs

Return to the Jenkins startpage, klick on the jenkins-debian-glue-source Jenkins job and select Build now. A few seconds later the job should be finished and automatically triggers the jenkins-debian-glue-binaries Jenkins job. This will take a while for its first run (setting up the cowbuilder environment for you).

Once the jenkins-debian-glue-binaries Jenkins job has finished as well you should have two Jenkins jobs marked as blue (AKA success):

Problems? Check out the build console log which should be available at http://$SERVER_IP:8080/job/jenkins-debian-glue-binaries/architecture=amd64/lastBuild/console

Well done - that's it! You have your own Debian packages of jenkins-debian-glue now.

Execute sudo repository_checker --list-repos jenkins-debian-glue to get a listing of Debian packages inside the jenkins-debian-glue repository, which is available in /srv/repository. Now you have everything to add your own Debian packages and enjoy the benefits of Continuous Integration.

Next Steps - Customization



Fork me on GitHub