Gitlab CI

Posted on | Edited on | In

目前使用的source code management tool 是gitlab, 除了行使git 的功能外，对每次 merge request 都做了额外的CI/CD操作，这里记录一下相关语法和总结 from course Continuous Delivery with GitLab

CI: code and feature integraion, combining updates into existing code base, testing with automation. CD: delivery can mean deployment, the process of building and deploying the app, for example, upload the object to somewhere that customer can download.

Gitlab uses pipelines to do both CI/CD, defined in .gitlab-ci.yml file at your branch.

Tips

[x] To navigate the source code in gitlab repo, try launch the Web IDE, will show you a structure tree on left side of the files. [x] Use snippet to share code or file block for issue solving, the same as gPaste. [x] To-do list is someone mentions you in some events. [x] Milestone is a goal that needs to track. [x] Merge request (pull request in github) after merged can auto close the issue, deponds on setting.

Setup self-managed Gitlab

You can experiment with gitlab community edition locally by bringing up a gitlab server through Vagrant. For example, Vagrantfile, there are 2 VMs, one VM for configuring docker gitlab runner:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# -*- mode: ruby -*-
# vi: set ft=ruby :

# server static ip
GITLAB_IP = "192.168.50.10"
# worker static ip
GITLAB_RUNNER_IP = "192.168.50.11"

Vagrant.configure("2") do |config|
  # gitlab server VM
  config.vm.define "server", primary: true do | server|
    server.vm.hostname = "gitlab"
    server.vm.box = "bento/ubuntu-16.04"
    ## private network
    server.vm.network "private_network", ip: GITLAB_IP

    server.vm.provider "virtualbox" do |v|
      v.memory = 2048
      v.cpus = 2
    end
  end

  # gitlab runner VM with docker installed
  config.vm.define "runner", primary: true do | runner|
    runner.vm.hostname = "runner"
    runner.vm.box = "bento/ubuntu-16.04"
    ## private network
    runner.vm.network "private_network", ip: GITLAB_RUNNER_IP

    runner.vm.provider "virtualbox" do |v|
      v.memory = 1024
      v.cpus = 2
    end
  end

end

Vagrant quick commands:

1
2
3
4
5
6
7
vagrant up
# ssh to server
vagrant ssh [gitlab]
# ssh to worker
vagrant ssh runner
# destroy uses
vagrant destroy -f

Install the packages references from there, but it uses enterprise edition, we use community edition.

1
2
3
4
5
6
7
8
9
10
11
12
# Update package manager and install prerequisites
sudo apt-get update
sudo apt-get install -y curl openssh-server ca-certificates
# we don't need email in this case, so skip it

# Set up gitlab apt repository
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash

# Install gitlab
# this is the IP address in vagrant file
# gitlab-ce is community edition
sudo EXTERNAL_URL="http://192.168.50.10" apt-get install gitlab-ce

After install, go to browser and hit http://192.168.50.10, reset root password and login as root with the reseted password.

Experiment

[1] Create a new project hello world (You can also create it by setting jenkins pipeline) Use root user to create a private project, check RAEDME added option.

[2] Create a admin user, so don’t need to use root user anymore. Grant the new user as admin, edit the password that will be used as temporary password next time you login. sign out and sign in again with new admin user.

[3] Setup SSH for your user The same process as setup SSH on github, go to setting -> SSH keys.

[4] Create new project under admin user, set as priviate scope.

[4] Create anthos vagrant VM as gitlab client To avoid messing up system git global configuration, then vagrant ssh and git clone the project.

Go to project dashboard, in the left menu: The CI/CD tab is what we will focus on The Operations tab is where gitlab integrate other systems in your stack, for example kubernetes. The Settings -> CI/CD is about configuration.

CI/CD

[x] SonarQube, code quality testing tool.

.gitlab-ci.yml 通过设计stage 搭配完成了both CI/CD 的操作。可以通过不同的条件判断，对特定的branch 进行不同的CI/CD. 每次MR 之前和之后都各有一个 pipeline，针对的是MR前后的branch. 设置了jenkins pipeline double-commit 到master branch, 因为如果需要修改gitlab-ci.yml 只会checked in 到 master中， 所以变化要在master中得到体现。

CI test levels, each of them is a stage in pipeline, should fail early and fail often.

  • syntax and linting
  • unit and integration
  • acceptance

Gitlab runner is similar to jenkins, support run on VM, bare metal system or docker container or kubernetes. Here we use docker, so install docker first, can reference here

Here we install docker on gitlab server VM. [x] You can spin up another VM with 2GB, install docker and run gitlab runner container there. But make sure the VM can ping each other, just like what I did in Vagrantfile.

This docker install is on Ubuntu, Centos or other linux distro please see different way to install docker:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
sudo apt-get update

sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

# add docker's official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# add stable repository
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

# install docker
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io

# verify install good
sudo docker run hello-world

Install docker gitlab runner, reference is here

1
2
3
4
5
6
# name is gitlab-runner
# -v: will create folder automatically
sudo docker run -d --name gitlab-runner --restart always \
    -v /srv/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest

Then register the runner to your gitlab project, go to gitlab project Settings -> CI/CD -> Runners expand to see the registeration token.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# later gitlab-runner is command
# register is argument
sudo docker exec -it gitlab-runner gitlab-runner register

# command prompt:
Enter the GitLab instance URL (for example, https://gitlab.com/):
# from runner expand
http://192.168.50.10/
Enter the registration token:
# from runner expand
K5G9S5e5wmcdoANUGLF4
Enter a description for the runner:
[5922b65a9261]: docker
Enter tags for the runner (comma-separated):
# gitlab-ci will refer this tag
docker-tag
Registering runner... succeeded                     runner=K5G9S5e5
Enter an executor: docker, docker-ssh, virtualbox, docker+machine, docker-ssh+machine, custom, parallels, shell, ssh, kubernetes:
docker
Enter the default Docker image (for example, ruby:2.6):
# this can be overrided later
alpine:latest

Then reload the gitlab runner page, you will see the registered runner is there, click runner name to see specific. This runner is locked to this project, but you can alter it (the edit icon right near runner).

Create .gitlab-ci.yml in your repo to specify the pipeline, if you create it on web IDE, you can choose a template for it, for example the bash template, more advanced syntax please see gitlab-ci doc:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
---
# will override the image alpine:latest above
image: busybox:latest

# global variable, used by ${CHART_NAME}
variables:
  CHART_NAME: xxxx
  VERSION_NUM: xxxx

# specify order or skip some stages
stages:
  - test
  - build
  - deploy

before_script:
  - echo "Before script section"
  - echo "For example you might run an update here or install a build dependency"
  - echo "Or perhaps you might print out some debugging details"

after_script:
  - echo "After script section"
  - echo "For example you might do some cleanup here"

# execute in order if no stages list
build1:
  # tags means run on the docker runner I installed above that taged as `docker-tag`
  tags:
    - docker-tag
  stage: build
  script:
    - echo "Do your build here"

test1:
  tags:
    - docker-tag
  stage: test
  script:
    - echo "Do a test here"
    - echo "For example run a test suite"

test2:
  tags:
    - docker-tag
  stage: test
  script:
    - echo "Do another parallel test here"
    - echo "For example run a lint test"

deploy1:
  tags:
    - docker-tag
  stage: deploy
  script:
    - echo "Do your deploy here"

In the Pipeline page, CI Lint is the tool can edit and validate the .gitlab-ci yaml file syntax. You can also use Settings -> CI/CD -> Environment variables expand to set the env variables.

[x] where is the run-dev-check.sh script hosted? it is git cloned from another repo.

1
2
3
script:
  - git clone -v $CLOUDSIMPLE_CI_REPO_URL
  - ci-cd/common-jobs/run-dev-check.sh
0%