Jenkins Build Pipeline project

Jenkins has a lot more tools than you can imagine to get the work done. Today we’ll use some of those tools to create a full build pipeline to generate notifications for creating a new AMI instance in aws, triggered via a Git Hub commit change.

So as for the steps we have to consider the following.

  1. How to get git hub commit triggers to Jenkins
  2. Create AWS instance and form new AMI with commit changes
  3. Trigger notifications to email groups

We are going to first setup the Build projects needed for this flow in Jenkins.

create 3 freestyle projects

  • Check_Aws
  • Packer_Build
  • Publish

Both Check_Aws and Packer_Build steps have to be configured as Git hub projects.

Check_Aws Build step creation

In Check_Aws project, you have to give credentials of git hub and make the Source code Management to git hub in project settings. Set polling rate in Build Triggers Section.

 

Then we have to set an script to check the AWS through an aws-cli command. For this we have to add a Build-Step.

From Add Build Step drop down in Build section of the project settings, select Execute Shell

Here i have used aws-cli to check for AMI instances which have a tag name set to a specific value. And if there are any existing matching AMI’s they will be deleted.

#!/bin/bash

echo "=============================Checking For Existing AMI's================================"

AMI=$(aws ec2 describe-images --region us-east-1 --filters "Name=tag:Name,Values=xxxxxxxxxx" --owner xxxxxxxxxxxx --query 'Images[*]' | grep ami | awk {'print $2'} | tr -d '", ' )

tempAMI=( $AMI )

if [ $tempAMI ]
then
 for currentAMI in "${tempAMI[@]}"
 do
 echo deleting $currentAMI
 aws ec2 deregister-image --image-id $currentAMI --region us-east-1
 echo Successfully deleted $currentAMI

done
else
 echo "No AMI to delete"
fi

 

Packer_Build step creation

As I have described in This article, set up the Packer_Build project to include git repo into its workspace and make sure you already have packer scripts to set up the ami included in the git repo. You’ll need Packer Plugin for Jenkins as well

After setting up the packer settings in the Packer_Builder project as a post build action, next you have to set another post build action to set the ami-id of the newly created AMI as an Environment variable to be accessed later in our publishing step.

For this we are going to use the Groovy Plugin for Jenkins to invoke an groovy script that can set up environment variables in the run time of the Jenkins environment. [NOTE: This is not equal to setting export variables in bash]

After installing grooving plugin, go to the Packer_Build project and select Add Post Build Action. Select Groovy Post Build. Add the following script so that it invokes an Script saved in your local machine to check for available aws AMI’s with a given tag name.

import jenkins.model.Jenkins

instance = Jenkins.getInstance()
globalNodeProperties = instance.getGlobalNodeProperties()
envVarsNodePropertyList = globalNodeProperties.getAll(hudson.slaves.EnvironmentVariablesNodeProperty.class)

newEnvVarsNodeProperty = null
envVars = null

if ( envVarsNodePropertyList == null || envVarsNodePropertyList.size() == 0 ) {
 newEnvVarsNodeProperty = new hudson.slaves.EnvironmentVariablesNodeProperty();
 globalNodeProperties.add(newEnvVarsNodeProperty)
 envVars = newEnvVarsNodeProperty.getEnvVars()
} else {
 envVars = envVarsNodePropertyList.get(0).getEnvVars()

}

def sout = new StringBuilder(), serr = new StringBuilder()
def proc = "/home/ubuntu/Packer/testsh.sh".execute()
proc.consumeProcessOutput(sout, serr)
proc.waitForOrKill(1000)

envVars.put("CURRENT_AMI_ID", sout.toString())

instance.save()

This groovy script will set up an Env variable called CURRENT_AMI_ID.  The script /home/ubuntu/Packer/testsh.sh would be as follows.

 

aws ec2 describe-images --region us-east-1 --filters "Name=tag:Name,Values=xxxxxxxxxxxxxxxxx" --owner xxxxxxxxxx --query 'Images[*]' | grep ami | awk {'print $2'} | tr -d '", '

 

When groovy build process starts, it will run the bash script and pass the result to an Env variable which can be accessed in Global Jenkins settings.

Publish Build step creation

Next we have to make the Publish project. we are going to use the Email-Ext plugin for publishing custom emails. Install the plugin in Manage Plugins->Manage Plugins in Jenkins settings

After Installing and restarting Jenkins, go to Manage Jenkins->Configure System

Go to Extended Email Notifications.

Add the SMTP settings as follows if you want to use gmail

SMTP server - smtp.gmail.com
Use SMTP Authentication - Ticked
Username - sender@gmail.com
Password - senderPassword
Use SSL - Ticked
SMTP port - 465

Then we are going to make a slight change in the email body. I’m going to inclde the Env variable which is the current AMI id set in our Jenkins global variable by the groovy script.

 

$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS:
New AMI - ${ENV,var="CURRENT_AMI_ID"}
Check console output at $BUILD_URL to view the results.


Goto project settings, in the post build section, click Add Post Build Action and select Editable Email Notification.

Keep the Default content and subjects as it is. Click On Advanced Settings.  In Triggers, add Failure or Success notifications. Click Advanced and set the emails that you need or simply set a email (eg:Developer)

 

After building these 3 separate build steps now we have to make the pipeline so each will trigger its downstream build step. In each of the 3 build steps we have to trigger each build step accordingly.

So Build step flow should be like  Check_Aws step -> Packer_Build -> Publish

For this to happen, we have to set Build triggers. Go to Packer_Build project settings. In Build Triggers section, set the Check_Aws step as the “Build After Project”
Then Go to Publish project settings and set build trigger to Packer_build

Pipeline View in Jenkins

Now we have successfully created and pipeline but nno clear view to see the full pipeline process. For this, install the Build Pipeline plugin to create a pipeline.

In the jenkins home, click + sign in views to create a new view.

Then select Build Pipeline View and give a name for your view. Make any changes in the settings preview and click save. This will add a new view layout with a pipeline view of your project. Build and see the flow of your Pipeline Build to create and notify the aws AMI using packer for a commit trigger in Git Hub.

Enjoy your new Jenkins pipeline…!!!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s