CI/CD pipeline with Docker

In this project, I will present some techniques, tools, and methodologies that are a crucial part of modern DevOps daily practices and cover the main parts of the Software Delivery Lifecycle. CI/CD, as continuous integration/continuous deployment, is a practice that automates the whole process of building, testing, and deploying code from development to production environments.

For the purpose of my project, I will use GitLab as the main platform for running CI/CD jobs, and together with Docker, I will obtain a powerful tool for my development workflow. Docker is the most popular containerization platform using virtualization to deliver software in packages known as containers.

Here is what I have prepared for the initial steps:
  • Linux machine with Git
  • Docker and Docker Compose installed locally to test how the site works (optional)
  • Accounts on GitLab.com and Docker hub.

To start, I will download a Hugo template for a static sample page from gohugo.io, install it, and organize the file structure in the local repository. The site even provides a sample docker-compose.yml file, which I will use immediately after creating my first project in GitLab - hugo. I clone the project locally, place all necessary files in it, then commit, and push.

compose.yml


After having the initial project files, I will create an account on Docker Hub to use it as a registry. Registry where the images for the templates I will push and use for my CI/CD will be stored. For this purpose, at the group level in GitLab, I will enter the secrets necessary for docker login, which are pre-created in Docker Hub.

Settings -> CI/CD -> Variables

Landscape


The logic for building docker templates is placed in a new repository - gitlab-ci-templates and it consists of a simple YAML file using Docker-in-Docker

docker.yml


    image: docker:24.0.5
    services: 
      - docker:24.0.5-dind 
    before_script:
      - docker login -u "$DOCKER_USER" -p "$DOCKER_PASS"
    stages:
      - build-and-push
    
    latest:
      stage: build-and-push
      script:
        - docker build -t ilchevst/${CI_PROJECT_TITLE}:latest .
        - docker push ilchevst/${CI_PROJECT_TITLE}:latest
      rules:
        - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    
    tag:
      stage: build-and-push
      script:
        - docker build -t ilchevst/${CI_PROJECT_TITLE}:latest .
        - docker push ilchevst/${CI_PROJECT_TITLE}:${CI_COMMIT_TAG}
      rules:
        - if: $CI_COMMIT_TAG
          when: on_success
        

It should be noted that CI/CD jobs have predefined variables that we can use to authenticate various GitLab components. In this case ${CI_PROJECT_TITLE} is the project name displayed in the web interface.
Here, I have combined the build and push stages into one but separated them according to the rule if: $CI_COMMIT_TAG depending on whether we will build an image with a Tag number or as "latest".

The only thing left is to create a Dockerfile from which our image will be built (I took a sample file for HUGO with included nginx from here: docker.hugomods.com) as well as the necessary .gitlab-ci.yml file for running the workflow:

.gitlab-ci.yml


    include:
    - project: 'portfolio3955934/gitlab-ci-templates'
      file: 'docker.yml'
    

All files so far are created locally in the root directory of my Git repository, which I will use as a central location and a crucial component of the Version Control System for managing and tracking changes of all my files and directories.

Save the changes, commit and push... Voilà! And I see that our pipeline is successful. We have now the logic for creating images stored on GitLab.

Landscape

We can also test with a tag directly from the hugo project:

Code -> Tags -> New tag  1.0.0

and get the following result:

Landscape

Of course, I use the official Semantic Versioning scheme everywhere, as it is good to use it to avoid building only latest images.

However, I realize that I want to create a hidden task .docker in docker.yml, where I want to place the before_script with docker login, to remove them from the stages and not be visible


    .docker:
      before_script:
        - docker login -u "$DOCKER_USER" -p "$DOCKER_PASS"
    latest:
      stage: build-and-push
      extends: .docker 
      
        

I also use extends in the stages to indicate that we have a hidden task .docker.

And so, we already have a simple environment, all files are created and reside in the repositories.
Finally, it remains only to test whether the images pushed to Docker Hub are built successfully and whether the site loads after running the container locally:

Landscape

Quick bash command check:

Landscape

And as we see, the image is downloaded, the container starts successfully, and our site is already loading locally.


As a goal for the next project, let's bootstrap a remote server on which we want to upload and host our site and deployment of everything necessary for it to work.