web-log/container/site/content/posts/jenkins/index.md

+++ categories = ["software"] tags = ["automation","docker","jenkins"] date = 2024-10-31T15:49:57Z description = "Running Jenkins in Docker and connecting it to Gitea as the foundation for automated development build and deployment pipelines." draft = false slug = "jenkins" title = "🐋 Configuring Development Pipeline with Jenkins & Docker" author = "nicholas" +++

I have a few Gitea git repositories that I host on my local network. Eventually I would like to be able to configure my server to automatically build and deploy the latest changes to these repositories to a development site. In this post, I will configure only the infrastructure that will make this possible.

First Steps

The first thing I do with any project that runs in a container is to give it its own directory. I create the Docker Compose configuration file that will describe the docker container.

nicholas@nas:/$ cd /hdd1/nicholas/docker/dockerfiles/
nicholas@nas:/.../$ mkdir jenkins && cd jenkins
nicholas@nas:/.../jenkins/$ nano docker-compose.yml

services:
  jenkins:
    image: jenkins/jenkins:jdk17
    container_name: jenkins
    hostname: jenkins
    restart: always
    volumes:
      - ./data:/var/jenkins_home/
    ports:
      - "2376:2376"
      - "8882:8080"
      - "50000:50000"

Explanation

• restart: always: Configures the container to always restart automatically if it stops unexpectedly. This keeps Jenkins running persistently.
• volumes: Mounts folders on my host to a folder inside the container:
  • ./data:/var/jenkins_home/: Maps the local data folder to /var/jenkins_home/ inside the container for data persistence across restarts (Jenkins stores all its configuration, plugins, and job data in /var/jenkins_home)
• ports: Maps ports on the host to ports inside the container:
  • 2376:2376: This port is typically used by Docker daemons for secure Docker client connections.
  • 8882:8080: Maps port 8080 inside the container (Jenkins’ web UI port) to port 8882 on the host. I access Jenkins at http://host:8882.
  • 50000:50000: Maps the Jenkins agent communication port (50000) for connecting with Jenkins agents.

Create a data folder for Jenkins to use as a bind mount

 mkdir data

Start the container

sudo docker compose up -d

This will start the container. Now I can access it at http://nas:8882, but in a few steps I have not described in this post, I have configured my nginx reverse proxy to forward requests at https://build.uuard.com to the underlying docker container at http://nas:8882. I can use these two addresses interchangeably – they point to the same container. Before I can do any more configuration, I need to complete the Getting Started section of the web UI, which is mostly is self-explanatory.

In my compose file, I mapped the host directory ./data to the container directory /var/jenkins_home. Therefore, I am able to read the password at ./data/secrets/initialAdminPassword

cd data/secrets && cat initialAdminPassword
7ca8db936d794e78a70df88a1b9c2c1h

Finishing Initial Setup

I selected 'Install suggested plugins'. I am only trying to get something working quickly, not optimizing. This step shows a very cool dashboard of the installation progress.

{{< video src="videos/setup.webm" width="100%"

}}

Installing Gitea Plugin

Since I am hosting my own version control software Gitea, as opposed to, for example, GitHub or GitLab, I will need to install and configure the Jenkins Gitea plugin, which is not installed by default via the 'Install suggested plugins' option that I selected in the step above. This does not complicate things much at all. The video below shows the simple installation process, and further down I show the configuration related to the Gitea plugin.

{{< video src="videos/jenkins_install_gitea_plugin.webm" width="100%"

}}

Configuring Gitea Service Account

The next step is to create a service account that Jenkins can use to make requests to Gitea API. I could use my personal Gitea account to create the token, which would simplify things somewhat, but this does not accord with best practices. This will make the setup more robust and extensible. The service user will belong to the same organization as the owner of the repository and will therefore have the same ability to read and write to the repository. I name the account service_account. I do not describe this step in any more detail.

Generating an Access Token

I sign in to the service account and generate an access token with only the minimum permissions necessary. I need to copy this access token and preserve it for use in the next step. The video below shows the process of creating an access token.

{{< video src="videos/gitea_generate_token.webm" width="100%"

}}

Add Gitea Credentials to Jenkins

I use the access token string generated in the step above to create a credential within Jenkins. This credential will be used in the next step. The video below shows the process of creating the gitea credential in Jenkins.

{{< video src="videos/jenkins_add_gitea_credential.webm" width="100%"

}}

Configure Jenkins Pipeline

I create a new Jenkins Item (Pipeline). The configuration options below instruct Jenkins to fetch the pipeline definition (the Jenkinsfile) from a source control management (SCM) system – in this case Gitea. Jenkins will look for a Jenkinsfile in the specified repository build_test.git and branch main. The build trigger for this pipeline is a post-commit hook. After each commit, a script runs that sends a request to Jenkins, which triggers it to check out the latest commit from the specified branch, and execute build steps according to the Jenkinsfile.

{{< video src="videos/jenkins_configure_pipeline.webm" width="100%"

}}

• Under ⚙️Build Triggers
  • Check the box ☑️ Poll SCM – leave poll schedule textarea empty
    • No schedules so will only run due to SCM changes if triggered by a post-commit hook
• Under 〰️Pipeline
  • Select Pipeline script from SCM
    • Select Git SCM
      • Enter Repository URL "https://git.uuard.com/build_test.git"
      • Select the Gitea credential created in the step above: "gitea"
      • Enter Branch Specifier "main"
      • Script path "Jenkinsfile"

Jenkinsfile / Pipeline Definition

The pipeline definition only has one stage, which I have named Checkout, which uses Jenkins' built-in command checkout to checkout a git repository defined in the scm variable. The scm variable contains the git SCM configuration that was defined in the Jenkins Pipeline Item above. The Jenkinsfile below resides in the git repository root directory.

pipeline {
    agent any
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
    }
    post {
        failure {
            echo "Build Failed"
        }
    }
}

The video below shows the process of pushing a commit to the repository, the git webhook being sent to Jenkins (invisibly, in the background) and Jenkins receiving the request and executing actions defined in the pipeline Jenkinsfile

{{< video src="videos/jenkins_trigger_build.webm" width="100%"

}}

Creating Build Agent / Jenkins Node

The next step is to create a dedicated build agent that will execute the steps outlined in a build pipeline. I name the node build_test_agent. I select Launch agent by connecting it to the controller. This option is selected because I intend to use a separate docker container to run an inbound Jenkins agent. My Jenkins setup will operate under the master-controller/agent-slave pattern where the master or controller server (My docker container running docker image jenkins/jenkins:jdk17) manages the scheduling of jobs, while the agents or slaves (docker container(s) running the docker image jenkins/inbound-agent) execute the jobs as directed by the master/controller. The secret and the name generated after the agent is created will be used to create the build agent in the step below. The video below shows the process of creating such a build agent.

{{< video src="videos/jenkins_create_build_agent.webm" width="100%"

}}

Configure Build Agent

Once again, I need to create another directory for a Docker Compose definition.

    nicholas@nas:/$ cd /hdd1/nicholas/docker/dockerfiles/
    nicholas@nas:/.../$ mkdir jenkins_agent && cd jenkins_agent
    nicholas@nas:/.../jenkins/$ nano docker-compose.yml

Create docker-compose.yml File

services:
  jenkins-agent:
    container_name: jenkins-agent
    hostname: jenkins-agent
    image: jenkins/inbound-agent
    init: true
    command: ["-url", "https://build.uuard.com", "${AGENT_SECRET}", "${AGENT_NAME}"]
    restart: always

The .env file holds the AGENT_SECRET and AGENT_NAME environment variables which I reference in the command passed to the Jenkins agent container.

AGENT_SECRET=9dc1911f3a151cb37227dcc05467053511b5892e717bf1029add503ae2ffc66b
AGENT_NAME=build_test_agent

Start Build Agent

    sudo docker compose up -d

This starts the container. The video below shows the process of starting the agent container. Notice the build_test_agent status changes from (offline) to Connected. This agent is now available to do work directed by the Jenkins controller.

{{< video src="videos/jenkins_start_build_agent.webm" width="100%"

}}

Done! This will give me a foundation on which to build out a development pipeline for any number of my software projects.