Building a CI Docker Pipeline Using Docker in Your Docker

docker jenkins gogs ci

Note: This isn’t up to date anymore, instead use img .

First of all – why should you want to build all the docker images on your own build server?

Complete control over the build process
You know when the images are built and how up-to-date they are
Use of private repositories and images
Because you can

This assumes that you already have control over a private docker registry. If that isn’t the case yet, you can just use the pre-built registry image using the following Makefile. Just be sure to edit the placeholders marked by ~~.

Note: Use tabs instead of spaces in Makefiles!

SHELL := /bin/bash

# The directory of this file
DIR := $(shell echo $(shell cd "$(shell  dirname "${BASH_SOURCE[0]}" )" && pwd ))

# If you use an nginx-proxy for TLS, you have to set this accordingly.
VIRTUALHOST := "~~YourRegistryDomain"
CONTAINER_NAME ?= "~~YourContainerName"

# Use the registry image
IMAGE_NAME ?= registry
VERSION ?= 2


# This will output the help for each task
# thanks to https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html
.PHONY: help

help: ## This help
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)

.DEFAULT_GOAL := help


# Docker commands
# Build the container
build: ## Build the container
	docker build --rm -t $(IMAGE_NAME) .

build-nc: ## Build the container without caching
	docker build --rm --no-cache -t $(IMAGE_NAME) .

run: ## Run container
	docker pull $(IMAGE_NAME):$(VERSION) && \
	docker run \
		-d \
		--name=$(CONTAINER_NAME) \
		--restart=always \
		-v $(DIR)/data:/var/lib/registry \
		-v $(DIR)/htpasswd:/htpasswd \
		-v $(DIR)/config.yml:/etc/docker/registry/config.yml \
		--expose 5000 \
		$(IMAGE_NAME):$(VERSION)

stop: ## Stop a running container
	docker stop $(CONTAINER_NAME)

remove: ## Remove a (running) container
	docker rm -f $(CONTAINER_NAME)

remove-image-force: ## Remove the latest image (forced)
	docker rmi -f $(IMAGE_NAME):$(VERSION)

The help message of the Makefile can be displayed with make.

As you can see in the run rule, the file config.yml gets mounted into the container. You should create this file along with the specified .htpasswd file as specified in the run command of the Makefile to disallow anonymous access. That being said, the files should be placed in the same folder the Makefile was saved in.

My config.yml looks like this (minus the placeholder for the secret directive):

version: 0.1
log:
  level: info
  fields:
    service: registry
storage:
  cache:
    layerinfo: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
auth:
  htpasswd:
    realm: basic-realm
    path: /htpasswd
http:
  addr: :5000
  secret: ~~YourHTTPSecret

You can then use make run to start your private registry. The folder $(DIR)/data will be used for persistent storage.

Now that the registry is up and running, it’s time to create a Jenkins instance in another container using this Dockerfile:

FROM jenkins/jenkins:lts

USER root

# Install the dependencies to use docker in the container
RUN apt-get update && \
    apt-get install -y \
        sudo \
        libltdl7 \
        libltdl-dev \
        make && \
    rm -rf /var/lib/apt/lists/*

# Allow sudo access for the jenkins user in order
# to build images
RUN echo "jenkins ALL=NOPASSWD: ALL" >> /etc/sudoers

USER jenkins

It’s worth mentioning that this Jenkins instance should’nt be publically exposed because of different reasons:

Jenkins user = root in the container. However, this is required to build docker containers automatically using docker-in-docker.
Jenkins is Jenkins and we don’t trust Jenkins security.

Personally, I use another Makefile to run Jenkins (as before - mind the placeholders marked by ~~):

SHELL := /bin/bash
VERSION ?= latest

# The directory of this file
DIR := $(shell echo $(shell cd "$(shell  dirname "${BASH_SOURCE[0]}" )" && pwd ))
REGISTRY_VIRTUALHOST := "~~YourRegistryDomain"
DOCKER_PATH := $(shell which docker)

IMAGE_NAME ?= ~~YourRegistryImage
CONTAINER_NAME ?= ~~YourContainerName

# This will output the help for each task
# thanks to https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html
.PHONY: help

help: ## This help
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)

.DEFAULT_GOAL := help

# Docker commands
# Build the container
build: ## Build the container
	docker build --rm -t $(IMAGE_NAME) .

build-nc: ## Build the container without caching
	docker build --rm --no-cache -t $(IMAGE_NAME) .

run: ## Run container
	docker pull $(IMAGE_NAME):$(VERSION) && \
	chown -R 1000 $DIR/data && \
	docker run \
		-d \
		--name=$(CONTAINER_NAME) \
		--restart=always \
		-v $(DIR)/data:/var/jenkins_home \
		-v /var/run/docker.sock:/var/run/docker.sock:ro \
		-v $(DOCKER_PATH):/usr/bin/docker:ro \
		--expose 8080 \
		-p 127.0.0.1:8080:8080 \
		--link $(REGISTRY_VIRTUALHOST) \
		$(IMAGE_NAME):$(VERSION)

stop: ## Stop a running container
	docker stop $(CONTAINER_NAME)

remove: ## Remove a (running) container
	docker rm -f $(CONTAINER_NAME)

remove-image-force: ## Remove the latest image (forced)
	docker rmi -f $(IMAGE_NAME):$(VERSION)

A few things are important here:

Docker has to be installed on the host of the Jenkins container. The docker socket and binary will be mounted into the jenkins container.
The jenkins container will be linked to the registry container for efficient data transfers - keep the container names consistent!
Jenkins will only be accessible by localhost on the machine. I use SSH port forwarding to access the Jenkins instance from remote: ssh -L <localPort>:localhost:<exposedJenkinsContainerPort> user@yourServer.

Using this, jenkins is accessible via localhost:<localPort> on my machine but not from the interwebz.

Side note: Don’t just use make remove on the jenkins container if you are currently logged in - data can be corrupted easily. Logout first.

That’s it! Now connect jenkins with your private gogs instance or whatever. Here’s a hacky example of a Jenkinsfile one could use to build images:

node {

   try {

      def app

         stage('Clone repository') {
            /* Let's make sure we have the repository cloned to our workspace */

            checkout scm
         }

      stage('Build image') {
         /* This builds the actual image; synonymous to
          * docker build on the command line */
         sh 'sudo make build-nc'

      }

      stage('Push image') {
         withCredentials([[$class: 'UsernamePasswordMultiBinding',
               // set the dockerhub credentials
               credentialsId: 'YourCredentialID',
               passwordVariable: 'REGISTRY_PASSWORD',
               usernameVariable: 'REGISTRY_USERNAME']]) {
            sh '''sudo docker login yourRegistryDomain -u ${REGISTRY_USERNAME} -p ${REGISTRY_PASSWORD}
            sudo docker push yourRegistryDomain/yourImageName:latest'''
         }
      }

      stage('Cleanup') {
         sh 'sudo make remove-image-force'

      }

   }
   catch(error) {
      // Send a telegram messsage
      sh 'curl -i -X GET "https://api.telegram.org/botYourBotToken/sendMessage?chat_id=YourChatID&text=failed"'
         error "Exception --> Aborting"
   }


   sh 'curl -i -X GET "https://api.telegram.org/botYourBotToken/sendMessage?chat_id=YourChatID&text=OK"'

}

Building a CI Docker Pipeline Using Docker in Your Docker

Information Leak in Docker

Docker Breakout Using X11

Easy Remote Pair Programming Using Docker and Tmux

Building a CI Docker Pipeline Using Docker in Your Docker

Related

Information Leak in Docker

Docker Breakout Using X11

Easy Remote Pair Programming Using Docker and Tmux