Remote Bootstrap and Portainer Installation

Project Overview

In the previous project, we ensured that we have a remote repository image that can be successfully pulled and loaded onto a local workstation. Now that we know our files are ready, we need to find a way to deploy them to our remote machine, where the Hugo site will be hosted.

    Here's what I plan to do:

  • Provision a remote machine on Hetzner Cloud with a pre-installed OS (Ubuntu 22.04) and set up a public SSH key for access from GitLab.
  • Bootstrap the machine using version control and bash scripts to install and prepare everything necessary up to this point.
  • Install Portainer and prepare it for the next step.
GitLab Repository Setup

First, I will create additional subgroups and organize my projects on the GitLab repository for my convenience. The website will be in a subgroup called "code", and the templates will be in a subgroup called "devops",

compose.yml

then, I will simply change the URL for the corresponding project in the .git/config file on my workstation. For example:

compose.yml

Then on GitLab, I create a Bootstrap group with a ubuntu-vm-bootstrap project inside it and clone it to my local repository using the already familiar method.

Bootstrap Project

For the Bootstrap project, I want to have a single .gitlab-ci file that executes a simple code to prepare the machine in the easiest way possible. The file will use bash commands to prepare the machine, install Docker Compose, and create the site directory.

.gitlab-ci.yml


    image: ubuntu:latest
    stages:
      - update
      - install
    before_script:
      - apt update -y
      - apt install -y ssh
      - mkdir -p /root/.ssh
      - chmod 700 /root/.ssh
      - cp ${SSH_KEY} /root/.ssh/id_rsa
      - chmod 600 /root/.ssh/id_rsa
    variables:
      HOST: 116.203.83.76
      USER: root
      SSH_STR: "${USER}@${HOST}"
      SSH_OPTIONS: "StrictHostKeyChecking no"

    update:
      stage: update
      script:
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'apt update -y'
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'apt upgrade -y'
                    
    docker:
      stage: install
      script:
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'apt install -y docker.io'
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'wget -O /bin/docker-compose https://github.com/docker/compose/releases/download/v2.29.2/docker-compose-linux-x86_64'
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'chmod +x /bin/docker-compose'
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'mkdir -p /apps'
      resource_group: apt
                    
    common-packages:
      stage: install
      script:
        - ssh -o "StrictHostKeyChecking no" $SSH_STR 'apt install -y git wget curl net-tools'
      resource_group: apt

I use Ubuntu image and in the before_script I have described the ssh logic (install, chmod, ssh_key) so that it can log on remotely, as there is no ssh installed on this image by default. Once this is possible, in the first stage, we have an update/upgrade, and in the second stage, we have install Docker, Docker Compose, and all necessary packages.

There is no need to write the user and the host everywhere so I have chosen to use the variable SSH_STR: "${USER}@${HOST}" to use it for all other SSH commands. I have also set StrictHostKeyChecking no - I'd like to skip the host key verification process here for now as in the current logic the job will fail. We cannot save the fingerprint at this stage initially on the runner.

Since there is a single Install stage with multiple separate installations using apt install, it's a good idea to make them run sequentially. To limit job concurrency, I will use resource_group to achieve this. By putting both jobs in the same resource_group, I will make them run sequentially, and they will not run in parallel. In this case, only one job starts, and other jobs wait until the resource_group is free.

From the previous project, we already know how to set up secrets on group level, and here again we'll save the ssh key in the variables on GitLab.

compose.yml

I am ready with the file, save it, and run git add, git commit, and git push.

After the pipeline runs successfully, we can, of course, verify on the remote machine that the corresponding directory has been created, and the installation of Docker, Docker Compose, and other packages has been successful.

Final Steps

Now that we finally have our bootstrap complete, in the next step I want to adopt a GitOps approach by installing Portainer and from it to manage everything we have prepared so far as well as every other application from now on.

Portainer is a lightweight management UI that allows you to easily manage your different Docker environments and gives you the ability to automate the deployment of applications based on instructions in Git repos.

In the existing Bootstrap repository, I will create another YAML file that describes what we need for Portainer. From the Portainer deployment documentation, I find instructions for Docker run, which I plan to modify for Docker Compose.

Here is the portainer-docker-compose.yml file:

                
    services:
    portainer:
      image: portainer/portainer-ce:latest
      ports:
        - 8000:8000
        - 9443:9443
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - portainer_data:/data
    volumes:
      portainer_data:
        driver: local

As you can see, there is HTTP and HTTPS ports exposed, volumes where the docker.sock is mounted (we should be careful with this one), and Portainer data.

In the .gitlab-ci.yml file, I need to add a new stage that describes the installation of Portainer:

                 
    portainer:
    stage: install-portainer
    script:
      - ssh -o "StrictHostKeyChecking no" $SSH_STR 'mkdir -p /apps/portainer'
      - ssh -o "StrictHostKeyChecking no" $SSH_STR 'mkdir -p /data'
      - scp -o "StrictHostKeyChecking no" portainer-docker-compose.yml $SSH_STR:/apps/portainer
      - ssh -o "StrictHostKeyChecking no" $SSH_STR 'mv /apps/portainer/portainer-docker-compose.yml /apps/portainer/docker-compose.yml'
      - ssh -o "StrictHostKeyChecking no" $SSH_STR 'cd /apps/portainer && docker-compose up -d'

This script creates a new directory, copies the portainer-docker-compose.yml file to the corresponding directory, renames it to docker-compose.yml, and finally starts the containers.

compose.yml

Now we have a working Portainer on our server. It works on port 9443 and we can access it through the browser to make a quick setup and explore it:

compose.yml
Next Steps

The goal of the next project will be to introduce GitOps by setting up Portainer to use our repository in GitLab. This way we will have full control using VCS to deploy our site and all other necessary applications.