Dev and Ops are no longer exclusively separate roles within the IT space. Today, they’re morphing into one cohesive method and opportunity which is reshaping the way that IT teams operate. Most would define DevOps as a movement, practice, or culture that aims to tie IT professionals and software developers together so that they can more efficiently streamline infrastructure changes and software delivery. Essentially, it’s rooted in the idea that building, testing and releasing software can run more smoothly and automatically if the appropriate team of professionals are working together.
In this article, we will demonstrate a Continuous Integration (CI) pipeline consisting of GitHub, Docker and Jenkins in order to support test, configuration and deploy a simple LAMP (Linux, Apache, MySQL, PHP) stack.
A Three Part Process
The CI workflow described in this article is composed of three steps. The developer first pushes a commit to GitHub, which in turn uses a webhook to notify Jenkins of the update. Jenkins can then pull the GitHub repository, build the Docker container which contains our stack and then run the test. If the test passes, Jenkins will push the code to the master branch.
Source Control With GitHub
Version control is a method for tracking and controlling changes within your source code. This concept is typically used as the base for the control system. By utilizing this method, you have the ability to create and customize multiple directories to track and control. These files are commonly called a repository.
Today, one of the most popular online version control systems is Github. Github provides services for hosting repositories online which you can access at any time, regardless of your location. In order to use Github as an online solution for hosting repositories, you need to know the basics of Git. Before you start using Github, you need to create an account which requires three simple steps. Once you log in, the first page you will see is your dashboard. As shown below, start by creating a new public or private repository.
Create new repository page
After clicking Create repository the following screen displays the multiple ways that you can view your local repository and how to send your existing repository to Github.
Quick setup page
You can use the remote origin URL (the text that has the format firstname.lastname@example.org:<USERNAME>/<REPO NAME>.git), when using the SSH or HTTPS protocol. If you choose to use the HTTPS protocol there is no additional setup required. Every time that you execute the push command on the remote origin, Git will ask for your Github account password. When the SSH protocol is used, you will have to generate the SSH key before using the push command, otherwise the Git has to execute the push.
A Pull request is a feature of many online source control solutions that allow you to obtain the updates that others have made to your repository. Once a pull request is created, the owner(s) of the repository will be able to review the changes, and based on the quality of the commit, approve or decline the changes.
If someone created the pull requests you will notice that the Pull requests tab on the repository page has number of pull requests that are in the pending state (ready for review).
Pull request tab in the repository page
After clicking, the list of open pull requests displays.
List of open pull requests
After clicking one of the created pull requests a Detail page displays, with options to merge, comment or decline the created pull request.
Detail about created pull request
The last step of committing the changes that you made on your local repository is to push to the master. Usually the master branch is the branch where the production ready code is located. In case that you don’t have pull request, pushing to the master is as simple as executing the command git push to the origin master, after adding and committing changes in the local repository.
Build and Deploy with Jenkins
In this section, we will describe how to configure jobs for building Docker containers with LAMP so that you can easily start your website and expose it to the world.. The first step is to show you how to create a Docker image which has a LAMP software bundle, and then we will explain how to use a ‘dockerized’ LAMP stack with Jenkins to build the site.
Don’t have Jenkins installed? Start here
Dockerizing the LAMP Stack
Docker allows developers, system administrators and others that are involved in the development to deploy applications in the containers which are under the host operating system. Docker requires almost no overhead when the underlying system and resources are efficiently used. The Dockerfile with additional files can be found in our GitHub repository.
The command for building the Docker container from Dockerfile is docker build -t <NAME> <PATH> (ex. docker build -t lamp ., when you are located in the folder where Dockerfile is present, or docker build -t lamp /docker/lamp). Instead of <NAME> we will use the LAMP as an image name, and path will reference the folder where the Dockerfile and the rest are located. This command will be used in the next section to configure Jenkins for building and deploying these Docker containers.
Jenkins and GitHub webhooks
The first step is to install the required plugin for integrating GitHub with Jenkins, configure it, and then at the time of creating jobs, we will use this integration to trigger jobs on each new Git commit.
Manage Jenkins section
Under the Manage Jenkins section, which is located on the left side menu, click Manage Plugins. Then find the Available tab and in the Filter field enter the GitHub plugin term. Select from the table GitHub plugin and install it.
Plugin manager section
Once Jenkins restarts, we need to enable access to the GitHub API. Under the Manage Jenkins page, click on Configure System and locate the GitHub section.
GitHub section on the Configure system page
Next, click Add GitHub Server button and then click GitHub server item. After that you will notice the new view as in the following image.
GitHub section view after clicking Add GitHub server
Leave the Manage hooks checked, and click the Advanced button on the bottom of this section. Under the new view you will notice that Manage additional GitHub actions button appears. Click it and select Convert login and password to token. Next, under the new view, select from the credentials drop-down box existing login. Or, if you don’t have any existing credentials, click on From login and password and enter your login information on GitHub.
GitHub section and converting username and password into the token for GitHub API
Now you will be able to select the newly created token from the Credentials drop-down and test it using Test connection button.
Now we’re ready to create the job which will do the work for us. Click on New Item on the left side menu, and enter the item name as desired job name. Then select the Freestyle project.
New item page
Now on the configuration job page, under the section Git, enter your Repository URL.
Git section on the configuration job page
Now that it’s set up, the next step is to trigger a Jenkins job when a change is pushed to the Github. As you can see below, enter and configure Build Triggers. In this example we check the option Build when a change in the code repository occurs.
Build triggers section on the configuration job page
Now we will enter shell scripts for building and starting the container. We will also add two shell scripts for stopping and removing existing containers in order to build images while avoiding conflict.
Build section on the configuration job page
After clicking Save, the Jenkins is ready to monitor your repository and deploy your website whenever you change something on the repository.
An inherent part of the integration process is running tests on the stack. In our case we will use PHPUnit, which is one of the popular tools for starting and initiating PHP unit tests.
PHPUnit will allow you to execute your tests where Clover PHP (check the pictured below) will generate reports. One approach for creating the job for tests is to create a job that will execute the PHPUnit command and then using post build actions to point on the PHPUnit output reports file which will clover PHP process. The PHPUnit tool first need to be installed on the server where is the Jenkins and then Clover PHP plugin is installed in same way how we install the GitHub plugin in section about GitHub webhooks.
Clover PHP coverage report
If you want to add testing phase in within your job, just under the Build section, click on Add build step button and select the Execute shell item from the dropdown menu. After that you will notice that Command field is appear under the Execute shell sub-section of the Build. Now in the Command field we will enter the phpunit command which will generate report files based on unit files which you created (in our example the tests folder contain all PHP test cases file, check next command).
phpunit --log-junit 'reports/unitreport.xml' --coverage-html 'reports/coverage' --coverage-clover 'reports/coverage/coverage.xml' tests/
Phpunit command for generating test reports
In the post build actions for generating the unit test report we can then use the option, Publish JUnit test result.
After saving the job, then it’s ready for using, where the reports for unit tests will be waiting for us once the build is done the build, under the name Clover Code Coverage and Test Result.
Example of the JUnit report chart
So you pulled the code using and then ran the test. If the tests pass and your version is ready to be deployed on your master branch the docker image can than be built and deployed on your site servers.
Leveraging the trio – Github, Jenkins and Docker is proving to be very valuable for DevOps teams. By leveraging the tight integration with source code control mechanisms such as Git, Jenkins can initiate a build process each time a developer commits his code. And streamline creation of the complete application application stack to support test, staging as well as production operations.