Continuous Integration (CI)

Continuous Integration(CI) is a software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run. The key goals of continuous integration are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

I will walk you through the baby steps of implementation process Continuous Integration(CI) of DevOps for Java EE and Maven based Spring Petclinic project as follows.

Pre requisites

  • Linux Ubuntu Operating System 14.04 LTS.
  • Java JDK/JRE
  • DBMS (MySQL in my case)

Step 1: Install Jenkins
# wget -q -O – https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add –
# sudo sh -c ‘echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’
# sudo apt-get update
# sudo apt-get install jenkins

By default, Jenkins listen on port 8080. Access this port with browser to start configuration.
If we start it locally, we can see it running under  http://localhost/IP:8080/  URL.

Just follow the instructions…

Edit above /var/lib/jenkins/secrets/intialAdminPasswrod and copy the password & paste it over here>Click to continue.

Double click on Install suggested plugin option and wait for couple of minutes which will takes you to ‘Create First Admin User’ Page.

Provide all the required User details > Click to save and Finish>Start using Jenkins. This will takes you to below welcome page.

Step 2: Configure Jenkins System
Install Git and Maven on Ubuntu and Configure these two along with JDK in Jenkins

Step (a). Install Git:
# apt-get install git
Verify git:  #  git –version                  //gives you output like  ‘git version 2.7.4’

Step (b). Install Maven:
# apt-get install maven          
Verify maven:  # mvn  –version      //gives you output like Apache Maven version 3.3.9 along with maven & Java paths.

Step (c). Configure Git, Maven and JDK in Jenkins
Now Go to Jenkins>Manage jenkins>Global Tool Configuration and provide the tools paths.

Step (d). Install Few more Plugins in Jenkins (configure them if require):
Manage Jenkins>Manage Plugins>Available>select Sonar Integration plugin, Role-Based Strategy Plugin, Mask Password plugin and restart Jenkins.

Step 3: Install MySql, Sonarqube, Sonar-runner in Ubuntu and Configure them to Jenkins:

a). Installation of Mysql:
# sudo apt-get -y install mysql-server-5.6

Now, login to MySQL through terminal to create Sonar Database:
# mysql -u root -p
Create the database and a user with permissions:
CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE USER ‘sonar’ IDENTIFIED BY ‘sonar’;
GRANT ALL ON sonar.* TO ‘sonar’@’%’ IDENTIFIED BY ‘sonar’;
GRANT ALL ON sonar.* TO ‘sonar’@’localhost’ IDENTIFIED BY ‘sonar’;
FLUSH PRIVILEGES;

Note: Post installation of Mysql,if you want to modify mysql.conf to do any changes like bind address in ubuntu 15.x/ubuntu 16.x+ on wards edit & save below file.

cd etc/mysql/mysql.conf.d/mysql.cnf
sudo nano mysql.cnf

b). Download & Install Sonarqube:
# wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-6.1.zip
# unzip sonarqube-6.1.zip

Edit sonar.properties (if you wish to use custom database for consistancy)
nano /opt/sonarqube-6.1/conf/sonar.properties with your favorite text editor, and modify it.

#MySQL DB settings:
sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance

#Web Server settings    // required irrespective of DB
The following settings allow you to run the server on page http://localhost:9000/sonar
sonar.web.host=0.0.0.0/127.0.0.1
sonar.web.context=/sonar
sonar.web.port=9000

Step c). Download & Install sonar-runner
# wget http://repo1.maven.org/maven2/org/codehaus/sonar/runner/sonar-runner-dist/2.4/sonar-runner-dist-2.4.zip
# unzip sonar-runner-dist-2.4.zip

Edit sonar-runner.properties
nano /opt/sonar-runner-2.4/conf/sonar-runner.properties in a text editor, and modify it as like below.

sonar.host.url=http://localhost:9000/sonar            //save>exit and restart sonar service

Configure Sonarqube and runner paths in Ubuntu
#nano /etc/environment
SONAR_HOME=/opt/sonarqube-6.1
SONAR_RUNNER_HOME=/opt/sonar-runner-2.4

Restart and Verify Sonarqube Installation:
# sudo /opt/sonar/bin/linux-x86-64/sonar.sh restart

Browse the url http://IP:9000/sonar to confirm its installation.

Now, Configure both Sonarqube & Sonar-runner in Jenkins: Manage Jenkins>Configure System and provide details as in below>save.

Also, Configure Sonar-Runner in Manage Jenkins>Global Tool Configuration.

Step 4:  Secure Jenkins (add users for access, if required)
The easiest way is to use Jenkins own user database. Create at least the user “Anonymous” with read access. Also create entries for the users you want to add/allow users to sign up on login page via Manage Jenkins>Global Security Select-allow users to sign up as well as  Role-Based Strategy (a plugin already installed)>save.

Create users: A Returning user/admin can create users from Manage Jenkins>Manage Users>Create User

First Time user will be redirected to create user screen post click to “create an account”

Manage & Assign Roles:
Go to Manage Jenkins> Manage & Assign Roles > Create the roles and assign the permissions from the Manage & Assign Roles as like in below

Step 5. Download & Install Nexus Repository & Configure
Nexus Setup:
Step a). Add a user for nexus
sudo adduser –home /opt/nexus –disabled-login –disabled-password nexus

Step b). Change into that user, move to the home directory and unpack your Nexus download (by a link or by this command  
#  wget www.sonatype.org/downloads/nexus-2.14.1-01-bundle.tar.gz   )
sudo su – nexus
cd
tar -xzf /home/USER/Downloads/nexus-2.14.1-01-bundle.tar.gz
Then, back to our normal user again, using ‘exit’ command.

Step c). Now we setup the init script:
sudo ln -s /opt/nexus/nexus-2.14.1-01/bin/nexus /etc/init.d/nexus
In the init script, make sure the following variables are changed to the right values:
NEXUS_HOME=”/opt/nexus/nexus-2.14.1-01″
RUN_AS_USER=nexus   // It will be 4 lines below to above variable in commented stuff
##make the above file executable
# cd /opt/nexus/nexus-2.14.1-01/bin
# sudo chmod a+x nexus
Next change ownership of the extracted nexus oss folder recursively to nexus user and nexus group by command
# sudo chown -R nexus:nexus /opt/nexus/nexus-2.14.1-01/
Now you can change to user nexus and start the nexus oss
su – nexus
cd /opt/nexus/nexus-2.14.1-01/bin
./nexus start
Type the url http://IP:8081/nexus/ & verify that application is live

Login: with admin/admin123 credentials

Step d ). Create a Repository:
Click to ‘+ Add’ drop down on top> Select ‘Hosted Repository’ and provide Repository ID (in my case it is ‘PC’) and Repository Name as ‘Petclinic’ leave to defaults and save.

Artifact Upload:
Click to Artifact Upload -in Petclinic window>Select Gav Parameters Provide the Group Id (Dev), Artifact (spring-petclinic), version (1.1), Packaging (zip/war/ear)>Click on Select Artifacts to Upload and browse to any text file and open>click on Add Artifact>Upload Artifact>ok. 

Now you can check updated folder structure on any button by clicking on any button of Petclinic window. 

Step 5: Now, It is time to Create Jobs in Jenkins
Create a Folder:
First Create a Folder named  PetClinic for instance, which will be specific to your project(you may have more projects in future).
Jenkins>New Item>Enter Name and Select Folder>ok>Save.

Similarly, create a Delivery pipeline view by clicking to + button on Jenkins Dashboard.

Job1:  Build:
Step a). Now Click to PetClinic Folder>New Item>Free Style Project>ok.

Step b).In the Opened page, General-Select-Delivery Pipeline configuration provide Stage Name as ‘Build’ and Task Name as ‘Compile’ and Select-This project is parameterized- select two String Parameters one after another from ‘Add Parameter’ Drop down and provide BUILD_LABEL & COMMIT_ID in each.

Step c). Select, Git in Source Code Management and provide your Repository URL: https://github.com/Rammohanrmc/spring-petclinic.git & click on add button-provide your git credentials here, select>save.

Step d). In Build Triggers-Add build step, Select, Invoke Top Level Maven goals-provide the Goal as
install -Dmaven.test.skip=true

Step e). Select, Trigger parameterized builds on other Project from ‘Add post-build action’ at bottom of page & provide info as in below.

Create Build Job:
save>Click to Build now-Build job will compile execute source code successfully.

Job2: UnitTests
Step a). Click to New Item-inside folder, Name it as UnitTests>scroll to bottom and provide the ‘Build’ job name in Copy from Textbox>ok

****Below are the changes from Build job as we created this UnitTests job by copying its configurations.

Step b). In General-Delivery Pipeline configuration, provide Stage Name as ‘Build’ and Task Name as ‘Unit Test’.

Step c). Click on Advanced-bottom right in General>select, Use custom workspace-provide Build job path. I.e; /var/lib/jenkins/workspace/PetClinic/Build  and select nothing from Source Code Management.

Step d). In Build Triggers-’Add build step’, Select, Invoke Top Level Maven goals-provide the Goal as
test

Step e). Select- Publish JUnit test result report from ‘Add post-build action’ & provide info as in below

Provide Test report XMLs as target/surefire-reports/*.xml

Step f). Inpost-build actions’ of ‘Build’ at the bottom of page, change the ‘Projects to Build’ to  Static Code Analysis>save.

Trigger the build of UnitTests job post job success, Click on Test Result for unit result.

Job3: Static Code Analysis
Step a). Click to New Item-inside folder, Name it as Static Code Analysis>scroll to bottom and provide the ‘UnitTests’ job name in ‘Copy from’ Textbox>ok

****Below are the changes from UnitTests job as we created this job by copying its configurations.

Step b). In General-Delivery Pipeline configuration, provide Stage Name as ‘Static Code Analysis’ and Task Name as ‘Code Quality Check’.

Step c). In Build Triggers-’Add build step’, Select, ‘Execute shell’ and provide below commands to remove existing sonar properties in project.
# Generally this 2 line script in not needed. It is specific to Petclinic, we are removing this undesired.
cd /var/lib/jenkins/workspace/PetClinic/Build
rm -rf sonar-project.properties

Step d). Again,  In Build Triggers-’Add build step’, Select, ‘Execute sonarqube scanner’ and provide below sonar properties to run static code analysis on project.

Select Execute Sonarqube Scanner & provide Analysis properties as in below
sonar.projectKey=PetClinic
sonar.projectName=PetClinic
sonar.projectVersion=1.0
sonar.sources=.
sonar.verbose=true

Step e). Inpost-build actions’ of ‘Build’ at the bottom of page, change the ‘Projects to Build’ to  Package>save.
Trigger the build of ‘Package’ job post job success,
Browse the URL: http://IP:9000/sonar/  to observe code quality metrics.

Job4: Package
Step a). Click to New Item-inside folder, Name it as Package >scroll to bottom and provide the ‘UnitTests’ job name in ‘Copy from’ Textbox>ok

****Below are the changes from ‘UnitTests’ job as we created this job by copying its configurations.

Step b). In General-Delivery Pipeline configuration, provide Stage Name as ‘Package’ and Task Name as ‘Packaging Source Code’.

Step c). Select-Mask passwords>click Add and provide Name as NexusUserName, password as your admin user name in my case admin again click Add and provide Name as NexusPassword, password as you admin user password in my case admin123.

Step d). In Build Triggers-’Add build step’, Select, ‘Execute shell’ and provide below commands to remove existing sonar properties in project.
zip -r PetClinic-1.1.zip * .

Step e). In Build Triggers-’Add build step’, Select, ‘Execute shell’ and provide below commands to to store package in Nexus repo artifacts.

curl -F “r=PC” -F “e=zip” -F “hasPom=false” -F “g=Dev” -F “a=spring-petclinic” -F “v=${BUILD_LABEL}” -F “p=zip” -F “file=@PetClinic-1.1.zip” -u ${NexusUserName}:${NexusPassword} http://172.25.151.173:8081/nexus/service/local/artifact/maven/content

Note: Make sure that you install zip in your Ubuntu Machine before running package job.

Step f). Inpost-build actions’ of ‘Build’ at the bottom of page, change the ‘Projects to Build’ to  Package>save.
Trigger the build of ‘Package’ job post job success,
Browse for package in nexus url http://IP:8081/nexus/ 

Finally, Observe Continuous Integration in the below delivery pipe line on Jenkins Dashboard.

Delivery Pipeline Full Screen view of CI:

deliverypipeline

Note: Here I just added ‘Approval’ job to the delivery pipeline as I want to keep the interest of the readers about our next Article of DevOps on ‘Continuous Delivery (CD)’.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s