DevOps 7 min read

Jenkins Pipeline Tutorial: Complete CI/CD Guide

Master Jenkins pipelines for CI/CD automation. Learn declarative and scripted pipelines, stages, parallel execution, and deployment strategies.

MR

Moshiour Rahman

#jenkins #ci-cd #devops #automation #pipeline #ai

Advertisement

What is Jenkins Pipeline?

Jenkins Pipeline is a suite of plugins that enables implementing continuous delivery pipelines as code. It provides an extensible set of tools for modeling delivery pipelines.

Pipeline vs Freestyle

FeatureFreestylePipeline
DefinitionUI-basedCode as Jenkinsfile
Version controlLimitedFull Git integration
ComplexitySimple jobsComplex workflows
ReusabilityCopy/pasteShared libraries
VisualizationBasicStage view

Setup

Install Jenkins with Docker

# docker-compose.yml
version: '3.8'

services:
  jenkins:
    image: jenkins/jenkins:lts
    privileged: true
    user: root
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - jenkins_home:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - JAVA_OPTS=-Djenkins.install.runSetupWizard=false

volumes:
  jenkins_home:

Required Plugins

  • Pipeline
  • Git
  • Docker Pipeline
  • Credentials
  • Blue Ocean (optional, better UI)

Declarative Pipeline

Basic Structure

// Jenkinsfile
pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                echo 'Building...'
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            steps {
                echo 'Testing...'
                sh 'mvn test'
            }
        }

        stage('Deploy') {
            steps {
                echo 'Deploying...'
                sh 'mvn deploy'
            }
        }
    }
}

Agent Options

pipeline {
    // Run on any available agent
    agent any

    // Run on agent with specific label
    agent { label 'linux' }

    // Run in Docker container
    agent {
        docker {
            image 'maven:3.8-openjdk-17'
            args '-v /root/.m2:/root/.m2'
        }
    }

    // No global agent (define per stage)
    agent none

    stages {
        stage('Build') {
            agent {
                docker { image 'node:18' }
            }
            steps {
                sh 'npm install'
            }
        }
    }
}

Environment Variables

pipeline {
    agent any

    environment {
        APP_NAME = 'my-application'
        APP_VERSION = '1.0.0'
        DOCKER_REGISTRY = 'docker.io/myuser'
        DEPLOY_ENV = "${params.ENVIRONMENT ?: 'dev'}"
    }

    stages {
        stage('Build') {
            environment {
                // Stage-specific variables
                BUILD_ID = "${env.BUILD_NUMBER}-${env.GIT_COMMIT?.take(7)}"
            }
            steps {
                echo "Building ${APP_NAME} version ${APP_VERSION}"
                echo "Build ID: ${BUILD_ID}"
                sh 'printenv | sort'
            }
        }
    }
}

Parameters

pipeline {
    agent any

    parameters {
        string(name: 'BRANCH', defaultValue: 'main', description: 'Branch to build')
        choice(name: 'ENVIRONMENT', choices: ['dev', 'staging', 'prod'], description: 'Deployment environment')
        booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run tests')
        password(name: 'API_KEY', description: 'API key for deployment')
    }

    stages {
        stage('Build') {
            steps {
                echo "Building branch: ${params.BRANCH}"
                echo "Target environment: ${params.ENVIRONMENT}"
            }
        }

        stage('Test') {
            when {
                expression { params.RUN_TESTS == true }
            }
            steps {
                sh 'mvn test'
            }
        }
    }
}

Credentials

pipeline {
    agent any

    environment {
        // Single credential
        DOCKER_CREDENTIALS = credentials('docker-hub-credentials')

        // Username/password separately
        DOCKER_USER = credentials('docker-username')
        DOCKER_PASS = credentials('docker-password')
    }

    stages {
        stage('Docker Login') {
            steps {
                // Using withCredentials block
                withCredentials([usernamePassword(
                    credentialsId: 'docker-hub-credentials',
                    usernameVariable: 'DOCKER_USER',
                    passwordVariable: 'DOCKER_PASS'
                )]) {
                    sh 'docker login -u $DOCKER_USER -p $DOCKER_PASS'
                }
            }
        }

        stage('Deploy with SSH') {
            steps {
                withCredentials([sshUserPrivateKey(
                    credentialsId: 'ssh-key',
                    keyFileVariable: 'SSH_KEY',
                    usernameVariable: 'SSH_USER'
                )]) {
                    sh '''
                        ssh -i $SSH_KEY $SSH_USER@server.example.com 'deploy.sh'
                    '''
                }
            }
        }
    }
}

Conditional Execution

When Directive

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }

        // Only on main branch
        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                sh './deploy-staging.sh'
            }
        }

        // Based on environment
        stage('Deploy to Production') {
            when {
                allOf {
                    branch 'main'
                    environment name: 'DEPLOY_TO_PROD', value: 'true'
                }
            }
            steps {
                sh './deploy-prod.sh'
            }
        }

        // Based on changeset
        stage('Build Frontend') {
            when {
                changeset 'frontend/**'
            }
            steps {
                sh 'cd frontend && npm run build'
            }
        }

        // Expression
        stage('Notify') {
            when {
                expression { currentBuild.result == 'SUCCESS' }
            }
            steps {
                echo 'Build successful!'
            }
        }
    }
}

Parallel Execution

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'mvn test -Dtest=*UnitTest'
                    }
                }

                stage('Integration Tests') {
                    steps {
                        sh 'mvn test -Dtest=*IntegrationTest'
                    }
                }

                stage('E2E Tests') {
                    agent {
                        docker { image 'cypress/included:latest' }
                    }
                    steps {
                        sh 'npm run test:e2e'
                    }
                }
            }
        }

        stage('Deploy') {
            steps {
                sh './deploy.sh'
            }
        }
    }
}

Post Actions

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
    }

    post {
        always {
            // Always execute
            echo 'Pipeline completed'
            cleanWs()  // Clean workspace
        }

        success {
            echo 'Build succeeded!'
            // Archive artifacts
            archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            // Publish test results
            junit 'target/surefire-reports/*.xml'
        }

        failure {
            echo 'Build failed!'
            // Send notification
            mail to: 'team@example.com',
                 subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
                 body: "Build failed: ${env.BUILD_URL}"
        }

        unstable {
            echo 'Build unstable (test failures)'
        }

        changed {
            echo 'Build status changed from previous build'
        }
    }
}

Complete CI/CD Pipeline

Java/Spring Boot Pipeline

pipeline {
    agent any

    tools {
        maven 'Maven-3.8'
        jdk 'JDK-17'
    }

    environment {
        APP_NAME = 'spring-boot-app'
        DOCKER_REGISTRY = 'docker.io/myuser'
        DOCKER_IMAGE = "${DOCKER_REGISTRY}/${APP_NAME}"
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
                script {
                    env.GIT_COMMIT_SHORT = sh(
                        script: 'git rev-parse --short HEAD',
                        returnStdout: true
                    ).trim()
                }
            }
        }

        stage('Build') {
            steps {
                sh 'mvn clean compile'
            }
        }

        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'mvn test'
                    }
                    post {
                        always {
                            junit 'target/surefire-reports/*.xml'
                        }
                    }
                }

                stage('Code Quality') {
                    steps {
                        withSonarQubeEnv('SonarQube') {
                            sh 'mvn sonar:sonar'
                        }
                    }
                }
            }
        }

        stage('Package') {
            steps {
                sh 'mvn package -DskipTests'
                archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    docker.build("${DOCKER_IMAGE}:${GIT_COMMIT_SHORT}")
                }
            }
        }

        stage('Push Docker Image') {
            steps {
                script {
                    docker.withRegistry('https://docker.io', 'docker-hub-credentials') {
                        docker.image("${DOCKER_IMAGE}:${GIT_COMMIT_SHORT}").push()
                        docker.image("${DOCKER_IMAGE}:${GIT_COMMIT_SHORT}").push('latest')
                    }
                }
            }
        }

        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                sh """
                    kubectl set image deployment/${APP_NAME} \
                        ${APP_NAME}=${DOCKER_IMAGE}:${GIT_COMMIT_SHORT} \
                        -n staging
                """
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            input {
                message 'Deploy to production?'
                ok 'Deploy'
            }
            steps {
                sh """
                    kubectl set image deployment/${APP_NAME} \
                        ${APP_NAME}=${DOCKER_IMAGE}:${GIT_COMMIT_SHORT} \
                        -n production
                """
            }
        }
    }

    post {
        always {
            cleanWs()
        }
        success {
            slackSend channel: '#deployments',
                      color: 'good',
                      message: "Deployment successful: ${APP_NAME} (${GIT_COMMIT_SHORT})"
        }
        failure {
            slackSend channel: '#deployments',
                      color: 'danger',
                      message: "Deployment failed: ${APP_NAME} - ${BUILD_URL}"
        }
    }
}

Node.js Pipeline

pipeline {
    agent {
        docker {
            image 'node:18-alpine'
        }
    }

    environment {
        npm_config_cache = 'npm-cache'
        CI = 'true'
    }

    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }

        stage('Lint') {
            steps {
                sh 'npm run lint'
            }
        }

        stage('Test') {
            steps {
                sh 'npm test -- --coverage'
            }
            post {
                always {
                    publishHTML([
                        allowMissing: false,
                        reportDir: 'coverage/lcov-report',
                        reportFiles: 'index.html',
                        reportName: 'Coverage Report'
                    ])
                }
            }
        }

        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }

        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh 'npm run deploy'
            }
        }
    }
}

Shared Libraries

Directory Structure

(root)
├── src/
│   └── org/
│       └── mycompany/
│           └── Utils.groovy
├── vars/
│   ├── buildJavaApp.groovy
│   └── deployToKubernetes.groovy
└── resources/
    └── templates/
        └── deployment.yaml

vars/buildJavaApp.groovy

def call(Map config = [:]) {
    pipeline {
        agent any

        tools {
            maven config.mavenVersion ?: 'Maven-3.8'
            jdk config.jdkVersion ?: 'JDK-17'
        }

        stages {
            stage('Build') {
                steps {
                    sh 'mvn clean compile'
                }
            }

            stage('Test') {
                steps {
                    sh 'mvn test'
                }
            }

            stage('Package') {
                steps {
                    sh 'mvn package -DskipTests'
                }
            }
        }
    }
}

Using Shared Library

// Jenkinsfile
@Library('my-shared-library') _

buildJavaApp(
    mavenVersion: 'Maven-3.8',
    jdkVersion: 'JDK-17'
)

Best Practices

Pipeline Tips

pipeline {
    agent any

    options {
        // Timeout for entire pipeline
        timeout(time: 30, unit: 'MINUTES')

        // Discard old builds
        buildDiscarder(logRotator(numToKeepStr: '10'))

        // Prevent concurrent builds
        disableConcurrentBuilds()

        // Timestamps in console output
        timestamps()

        // Skip checkout if not needed
        skipDefaultCheckout()
    }

    stages {
        stage('Build') {
            options {
                // Stage-specific timeout
                timeout(time: 10, unit: 'MINUTES')
                retry(3)
            }
            steps {
                sh 'mvn clean package'
            }
        }
    }
}

Summary

FeatureDescription
DeclarativeSimpler syntax, opinionated
ScriptedFull Groovy power, flexible
StagesLogical groupings of steps
ParallelConcurrent execution
PostCleanup and notifications

Jenkins Pipeline enables powerful, maintainable CI/CD workflows defined as code.

Advertisement

MR

Moshiour Rahman

Software Architect & AI Engineer

Share:
MR

Moshiour Rahman

Software Architect & AI Engineer

Enterprise software architect with deep expertise in financial systems, distributed architecture, and AI-powered applications. Building large-scale systems at Fortune 500 companies. Specializing in LLM orchestration, multi-agent systems, and cloud-native solutions. I share battle-tested patterns from real enterprise projects.

Related Articles

DevOps

GitHub Actions CI/CD: Automate Your Development Workflow

Master GitHub Actions for CI/CD pipelines. Learn workflows, jobs, actions, secrets management, and deploy to any cloud platform automatically.

DevOps

GitHub Actions CI/CD Pipeline: Complete Tutorial

Build automated CI/CD pipelines with GitHub Actions. Learn workflows, jobs, actions, and deploy applications automatically with practical examples.

DevOps

Stop Wrestling YAML: How to Deploy 50 AI Models with Python Loops

Infrastructure as Code shouldn't be a copy-paste nightmare. Learn how to use Pulumi and Python to programmatically deploy scalable AI infrastructure without the YAML fatigue.

Comments

Comments are powered by GitHub Discussions.

Configure Giscus at giscus.app to enable comments.