Continuous Delivery (CD)

Continuous Delivery (CD) expands upon continuous integration by deploying all code changes to a testing environment after the build stage. When continuous delivery is implemented in place, we will always have a Deployment-ready build artifact that has passed through a standardised test process.

I am going to give a generic logical flow of a Delivery Pipeline where everything gets automated for seamless delivery. However, this flow may vary from organisation to organisation as per the requirement.

Let’s assume that we have 3 deployment environments (Development, Staging/QA and Production), then a typical delivery pipeline would have the following steps:

  1. Developer develop and commits the code changes to Version Control System tools like Git.
  2. Continuous Integration/Delivery automation servers like Jenkins pulls this code from the repository using the Git like plugin and build it using build tools like Ant or Maven or MSBuild and runs the unit tests.
  3. After this, same code base passes through static code analysis, An artifact is created, with a unique version number, and will be published to an artifact repository tools like Nexus.
  4. Configuration management tools like Chef provisions testing environment & deploys the app & then Jenkins, chef together releases the same code onto different Test environments on which testing is done using test automation tools like Selenium.
  5. Once the code is tested, the same code will be send for deployment on to the production server at a defined schedule.
  6. After deployment, it is continuously monitored by the tools like Nagios.

Achieving Continuous Deployment (CD) as part of DevOps Practices:
We can achieve CD in 2-ways depending on the requirement.
1. Using Jenkins alone without using any Configuration Management Tools like Puppet/Chef/Ansible.
2. Using Jenkins and Configuration Management Tools like Puppet/Chef/Ansible.

Lets see both one by one with a Java Application with Maven (you may use .Net with MS Build)…

Prerequisites
For the next steps, I assume, you have a a Jenkins instance and a remote Tomcat Server up and running with administrator privileges.
1. Continuous Deployment with Jenkins & without using Chef:  By deploying war file from Jenkins to remote Tomcat Server of Staging/QA environment .

Step 1. Generate & Archive the Artifacts in a Package job to produce a war/ear file:
1st Create job called package> ‘In Post Build actions’ of your build job>Select Archive Artifacts- and provide, Files to archive as  **/*.war>save>build.

Step 2. Install required plugins:  Install copy artifact & deploy to container plugins.  Also install & configure tomcat and edit tomcat users file in conf folder of desired QA/Stage environment  to add a user with the role ‘manager-script’ in order to accept remote deployments.
Ex: <role rolename=”manager-script”/>
<user username=”admin” password=”tomcat” roles=”manager-gui, manager-script”/>

Step 3. Deploy the archived Artifact to defined environment: In the next job, named DeployToStage, pick up the archived war artifact and deploy it to a QA/Stage env.
a). 1st create a job called DeployToStage and then add Build Step-select copy artifacts from another project & provide the Project Name and Artifacts to Copy.
b). “Post-build Actions”. Select the option “Deploy war/ear to a container” from the “Add post-build action” dropdown button. Fill  war file, path and tomcat details>save>run.

Now, you can browse to your application in Staging environment via url; http://172.25.150.40:8080.

In the next episode, we are going to see how can we achieve Continuous Deployment (CD) using Jenkins and a Configuration Management Tool called Chef.

DevOps

DevOps is a set of practices that emphasizes the communication and collaboration of software Developers, Testers, Operations professionals and other stakeholders while automating the process of software delivery and infrastructure changes. It aims at establishing a culture and environment where building, testing, and releasing software can happen quickly, frequently, and more reliably.

devopsoverview

DevOps Tool-chain

Because DevOps is a cultural shift and collaboration between development, operations and testing, there is no single DevOps tool, rather a set or “DevOps toolchain” consisting of multiple tools. Generally, DevOps tools fit into one or more of these categories, which is reflective of the software development and delivery process.

Code – Code development and review, Version control tools, code merging

Build – Continuous integration tools, build status

Test – Test and results determine performance

Package – Artifact repository, application pre-deployment staging

Release – Change management, release approvals, release automation

Configure – Infrastructure configuration and management, Infrastructure as Code tools

Monitor – Applications performance monitoring, end user experience

Though there are many tools available, certain categories of them are essential in the DevOps tool chain setup for use in an organization.

Tools such as Docker (containerization), Jenkins (Continuous Integration), Chef (Infrastructure as Code) and Vagrant (Virtualization Platform) among many others are often used and discussed.

In DevOps, Continuous Integration (CI), Continuous Delivery (CD) and Continuous Testing (CT) are 3 key aspects which are briefed below.

Continuous Integration (CI): Is a software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run. The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

Continuous Delivery (CD): Is a software development practice where code changes are automatically built, tested, and prepared for a release to production. It expands upon continuous integration by deploying all code changes to a testing environment and/or a production environment after the build stage. When continuous delivery is implemented properly, developers will always have a Deployment-ready build artifact that has passed through a standardized test process.

Continuous Testing (CT): Is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

What is the difference between Continuous Delivery and Continuous Deployment:

cdeploy

Continuous Delivery (CD): Automates the entire software release process. Every revision that is committed, triggers an automated flow that builds, tests, and then stages the update. The final decision to deploy to a live production environment is triggered by a developer/release manager.

Continuous Deployment: Is a step further to CD, with Continuous Deployment, revisions are deployed to a production environment automatically without explicit approval from a developer, making the entire software release process automated.

Advantages of DevOps:
  • Quick to Market.
  • Reliability in Delivery (no human errors)
  • Scale at ease (via configuration management tools).
  • Improved Collaboration (between dev & ops reduces risk by sharing work).
  • Secure.

Continuous Integration (CI)

Continuous Integration(CI) is a software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run. The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

I will walk you through the baby steps of implementation process Continuous Integration(CI) of DevOps for Java EE and Maven based Spring Petclinic project as follows.

Pre requisites

  • Linux Ubuntu Operating System 14.04 LTS.
  • Java JDK/JRE
  • DBMS (MySQL in my case)

Step 1: Install Jenkins
# wget -q -O – https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add –
# sudo sh -c ‘echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’
# sudo apt-get update
# sudo apt-get install jenkins

By default, Jenkins listen on port 8080. Access this port with browser to start configuration.
If we start it locally, we can see it running under  http://localhost/IP:8080/  URL.

Just follow the instructions…

Edit above /var/lib/jenkins/secrets/intialAdminPasswrod and copy the password & paste it over here>Click to continue.

Double click on Install suggested plugin option and wait for couple of minutes which will takes you to ‘Create First Admin User’ Page.

Provide all the required User details > Click to save and Finish>Start using Jenkins. This will takes you to below welcome page.

Step 2: Configure Jenkins System
Install Git and Maven on Ubuntu and Configure these two along with JDK in Jenkins

Step (a). Install Git:
# apt-get install git
Verify git:  #  git –version                  //gives you output like  ‘git version 2.7.4’

Step (b). Install Maven:
# apt-get install maven          
Verify maven:  # mvn  –version      //gives you output like Apache Maven version 3.3.9 along with maven & Java paths.

Step (c). Configure Git, Maven and JDK in Jenkins
Now Go to Jenkins>Manage jenkins>Global Tool Configuration and provide the tools paths.

Step (d). Install Few more Plugins in Jenkins (configure them if require):
Manage Jenkins>Manage Plugins>Available>select Sonar Integration plugin, Role-Based Strategy Plugin, Mask Password plugin and restart Jenkins.

Step 3: Install MySql, Sonarqube, Sonar-runner in Ubuntu and Configure them to Jenkins:

a). Installation of Mysql:
# sudo apt-get -y install mysql-server-5.6

Now, login to MySQL through terminal to create Sonar Database:
# mysql -u root -p
Create the database and a user with permissions:
CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE USER ‘sonar’ IDENTIFIED BY ‘sonar’;
GRANT ALL ON sonar.* TO ‘sonar’@’%’ IDENTIFIED BY ‘sonar’;
GRANT ALL ON sonar.* TO ‘sonar’@’localhost’ IDENTIFIED BY ‘sonar’;
FLUSH PRIVILEGES;

Note: Post installation of Mysql,if you want to modify mysql.conf to do any changes like bind address in ubuntu 15.x/ubuntu 16.x+ on wards edit & save below file.

cd etc/mysql/mysql.conf.d/mysql.cnf
sudo nano mysql.cnf

b). Download & Install Sonarqube:
# wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.1.zip
# unzip sonarqube-6.1.zip

Edit sonar.properties (if you wish to use custom database for consistancy)
nano /opt/sonarqube-6.1/conf/sonar.properties with your favorite text editor, and modify it.

#MySQL DB settings:
sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance

#Web Server settings    // required irrespective of DB
The following settings allow you to run the server on page http://localhost:9000/sonar
sonar.web.host=0.0.0.0/127.0.0.1
sonar.web.context=/sonar
sonar.web.port=9000

Step c). Download & Install sonar-runner
# wget http://repo1.maven.org/maven2/org/codehaus/sonar/runner/sonar-runner-dist/2.4/sonar-runner-dist-2.4.zip
# unzip sonar-runner-dist-2.4.zip

Edit sonar-runner.properties
nano /opt/sonar-runner-2.4/conf/sonar-runner.properties in a text editor, and modify it as like below.

sonar.host.url=http://localhost:9000/sonar            //save>exit and restart sonar service

Configure Sonarqube and runner paths in Ubuntu
#nano /etc/environment
SONAR_HOME=/opt/sonarqube-6.1
SONAR_RUNNER_HOME=/opt/sonar-runner-2.4

Restart and Verify Sonarqube Installation:
# sudo /opt/sonar/bin/linux-x86-64/sonar.sh restart

Browse the url http://IP:9000/sonar to confirm its installation.

Now, Configure both Sonarqube & Sonar-runner in Jenkins: Manage Jenkins>Configure System and provide details as in below>save.

Also, Configure Sonar-Runner in Manage Jenkins>Global Tool Configuration.

Step 4:  Secure Jenkins (add users for access, if required)
The easiest way is to use Jenkins own user database. Create at least the user “Anonymous” with read access. Also create entries for the users you want to add/allow users to sign up on login page via Manage Jenkins>Global Security Select-allow users to sign up as well as  Role-Based Strategy (a plugin already installed)>save.

Create users: A Returning user/admin can create users from Manage Jenkins>Manage Users>Create User

First Time user will be redirected to create user screen post click to “create an account”

Manage & Assign Roles:
Go to Manage Jenkins> Manage & Assign Roles > Create the roles and assign the permissions from the Manage & Assign Roles as like in below

Step 5. Download & Install Nexus Repository & Configure
Nexus Setup:
Step a). Add a user for nexus
sudo adduser –home /opt/nexus –disabled-login –disabled-password nexus

Step b). Change into that user, move to the home directory and unpack your Nexus download (by a link or by this command  
#  wget www.sonatype.org/downloads/nexus-2.14.1-01-bundle.tar.gz   )
sudo su – nexus
cd
tar -xzf /home/USER/Downloads/nexus-2.14.1-01-bundle.tar.gz
Then, back to our normal user again, using ‘exit’ command.

Step c). Now we setup the init script:
sudo ln -s /opt/nexus/nexus-2.14.1-01/bin/nexus /etc/init.d/nexus
In the init script, make sure the following variables are changed to the right values:
NEXUS_HOME=”/opt/nexus/nexus-2.14.1-01″
RUN_AS_USER=nexus   // It will be 4 lines below to above variable in commented stuff
##make the above file executable
# cd /opt/nexus/nexus-2.14.1-01/bin
# sudo chmod a+x nexus
Next change ownership of the extracted nexus oss folder recursively to nexus user and nexus group by command
# sudo chown -R nexus:nexus /opt/nexus/nexus-2.14.1-01/
Now you can change to user nexus and start the nexus oss
su – nexus
cd /opt/nexus/nexus-2.14.1-01/bin
./nexus start
Type the url http://IP:8081/nexus/ & verify that application is live

Login: with admin/admin123 credentials

Step d ). Create a Repository:
Click to ‘+ Add’ drop down on top> Select ‘Hosted Repository’ and provide Repository ID (in my case it is ‘PC’) and Repository Name as ‘Petclinic’ leave to defaults and save.

Artifact Upload:
Click to Artifact Upload -in Petclinic window>Select Gav Parameters Provide the Group Id (Dev), Artifact (spring-petclinic), version (1.1), Packaging (zip/war/ear)>Click on Select Artifacts to Upload and browse to any text file and open>click on Add Artifact>Upload Artifact>ok. 

Now you can check updated folder structure on any button by clicking on any button of Petclinic window. 

Step 5: Now, It is time to Create Jobs in Jenkins
Create a Folder:
First Create a Folder named  PetClinic for instance, which will be specific to your project(you may have more projects in future).
Jenkins>New Item>Enter Name and Select Folder>ok>Save.

Similarly, create a Delivery pipeline view by clicking to + button on Jenkins Dashboard.

Job1:  Build:
Step a). Now Click to PetClinic Folder>New Item>Free Style Project>ok.

Step b).In the Opened page, General-Select-Delivery Pipeline configuration provide Stage Name as ‘Build’ and Task Name as ‘Compile’ and Select-This project is parameterized- select two String Parameters one after another from ‘Add Parameter’ Drop down and provide BUILD_LABEL & COMMIT_ID in each.

Step c). Select, Git in Source Code Management and provide your Repository URL: https://github.com/Rammohanrmc/spring-petclinic.git & click on add button-provide your git credentials here, select>save.

Step d). In Build Triggers-Add build step, Select, Invoke Top Level Maven goals-provide the Goal as
install -Dmaven.test.skip=true

Step e). Select, Trigger parameterized builds on other Project from ‘Add post-build action’ at bottom of page & provide info as in below.

Create Build Job:
save>Click to Build now-Build job will compile execute source code successfully.

Job2: UnitTests
Step a). Click to New Item-inside folder, Name it as UnitTests>scroll to bottom and provide the ‘Build’ job name in Copy from Textbox>ok

****Below are the changes from Build job as we created this UnitTests job by copying its configurations.

Step b). In General-Delivery Pipeline configuration, provide Stage Name as ‘Build’ and Task Name as ‘Unit Test’.

Step c). Click on Advanced-bottom right in General>select, Use custom workspace-provide Build job path. I.e; /var/lib/jenkins/workspace/PetClinic/Build  and select nothing from Source Code Management.

Step d). In Build Triggers-’Add build step’, Select, Invoke Top Level Maven goals-provide the Goal as
test

Step e). Select- Publish JUnit test result report from ‘Add post-build action’ & provide info as in below

Provide Test report XMLs as target/surefire-reports/*.xml

Step f). Inpost-build actions’ of ‘Build’ at the bottom of page, change the ‘Projects to Build’ to  Static Code Analysis>save.

Trigger the build of UnitTests job post job success, Click on Test Result for unit result.

Job3: Static Code Analysis
Step a). Click to New Item-inside folder, Name it as Static Code Analysis>scroll to bottom and provide the ‘UnitTests’ job name in ‘Copy from’ Textbox>ok

****Below are the changes from UnitTests job as we created this job by copying its configurations.

Step b). In General-Delivery Pipeline configuration, provide Stage Name as ‘Static Code Analysis’ and Task Name as ‘Code Quality Check’.

Step c). In Build Triggers-’Add build step’, Select, ‘Execute shell’ and provide below commands to remove existing sonar properties in project.
# Generally this 2 line script in not needed. It is specific to Petclinic, we are removing this undesired.
cd /var/lib/jenkins/workspace/PetClinic/Build
rm -rf sonar-project.properties

Step d). Again,  In Build Triggers-’Add build step’, Select, ‘Execute sonarqube scanner’ and provide below sonar properties to run static code analysis on project.

Select Execute Sonarqube Scanner & provide Analysis properties as in below
sonar.projectKey=PetClinic
sonar.projectName=PetClinic
sonar.projectVersion=1.0
sonar.sources=.
sonar.verbose=true

Step e). Inpost-build actions’ of ‘Build’ at the bottom of page, change the ‘Projects to Build’ to  Package>save.
Trigger the build of ‘Package’ job post job success,
Browse the URL: http://IP:9000/sonar/  to observe code quality metrics.

Job4: Package
Step a). Click to New Item-inside folder, Name it as Package >scroll to bottom and provide the ‘UnitTests’ job name in ‘Copy from’ Textbox>ok

****Below are the changes from ‘UnitTests’ job as we created this job by copying its configurations.

Step b). In General-Delivery Pipeline configuration, provide Stage Name as ‘Package’ and Task Name as ‘Packaging Source Code’.

Step c). Select-Mask passwords>click Add and provide Name as NexusUserName, password as your admin user name in my case admin again click Add and provide Name as NexusPassword, password as you admin user password in my case admin123.

Step d). In Build Triggers-’Add build step’, Select, ‘Execute shell’ and provide below commands to remove existing sonar properties in project.
zip -r PetClinic-1.1.zip * .

Step e). In Build Triggers-’Add build step’, Select, ‘Execute shell’ and provide below commands to to store package in Nexus repo artifacts.

curl -F “r=PC” -F “e=zip” -F “hasPom=false” -F “g=Dev” -F “a=spring-petclinic” -F “v=${BUILD_LABEL}” -F “p=zip” -F “file=@PetClinic-1.1.zip” -u ${NexusUserName}:${NexusPassword} http://172.25.151.173:8081/nexus/service/local/artifact/maven/content

Note: Make sure that you install zip in your Ubuntu Machine before running package job.

Step f). Inpost-build actions’ of ‘Build’ at the bottom of page, change the ‘Projects to Build’ to  Package>save.
Trigger the build of ‘Package’ job post job success,
Browse for package in nexus url http://IP:8081/nexus/ 

Finally, Observe Continuous Integration in the below delivery pipe line on Jenkins Dashboard.

Delivery Pipeline Full Screen view of CI:

deliverypipeline

Note: Here I just added ‘Approval’ job to the delivery pipeline as I want to keep the interest of the readers about our next Article of DevOps on ‘Continuous Delivery (CD)’.