[A Cloud Guru, Linux Academy] Внедрение полного конвейера CI/CD [RUS, 2020]


07. Оркестрация

Делаю все с 0. На виртуалках.


01. Поднимаю локальный kubernetes кластер

Разворачиваю kubernetes в виртуалках.


02. Поднимаю в виртуалке Jenkis

$ mkdir ~/vagrant-jenkins && cd ~/vagrant-jenkins


Создаю Vagrantfile для виртуалки


$ cat <<EOF >> Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

ENV['VAGRANT_NO_PARALLEL'] = 'yes'

Vagrant.configure(2) do |config|
  config.vm.box = "ubuntu/focal64"
  config.hostmanager.enabled = true
  config.hostmanager.include_offline = true

  config.vm.provider "virtualbox" do |vb|
    vb.memory = "4096"
    vb.cpus = 2
  end

  config.vm.define "jenkins.k8s" do |c|
    c.vm.hostname = "jenkins.k8s"
    c.vm.network "private_network", ip: "192.168.0.5"
  end
end
EOF


$ vagrant up


$ vagrant ssh jenkins.k8s


$ sudo apt update
$ sudo apt upgrade -y


$ sudo apt install -y \
    openssh-server \
    rar unrar-free \
    unzip


Создаю пользователя “jenkins”

$ sudo su  -
# adduser --disabled-password --gecos "" jenkins
# usermod -aG sudo jenkins
# passwd jenkins


Предоставляю возможность подключения по SSH

# sed -i "s/.*PasswordAuthentication.*/PasswordAuthentication yes/g" /etc/ssh/sshd_config

# service sshd reload


Разрешаю выполнение команд sudo без пароля

# vi /etc/sudoers


%sudo   ALL=(ALL:ALL) ALL

меняю на:

#%sudo   ALL=(ALL:ALL) ALL
%sudo   ALL=(ALL:ALL) NOPASSWD:ALL


Устанавливаю JDK8

Устанавливаю Gradle

Устанавливаю Docker

Устанавливаю Jenkins


$ sudo usermod -aG docker jenkins
$ sudo systemctl restart jenkins
$ sudo systemctl restart docker


http://192.168.0.5:8080/


Генерация ключа для работы с GitHub

$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -q -N ""

$ cat ~/.ssh/id_rsa.pub


Вставляем на GitHub


GitHub -> Settings -> SSH and GPG keys



Создаю Credentials

Manage Jenkins -> Credentials


Jenkins


Проверяю возможность работы с kubernetes


$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local/bin/

$ mkdir -p ~/.kube

// root password: kubeadmin
$ scp [email protected]:/etc/kubernetes/admin.conf ~/.kube/config


$ kubectl get nodes
NAME         STATUS   ROLES                  AGE    VERSION
master.k8s   Ready    control-plane,master   127m   v1.20.1
node1.k8s    Ready    <none>                 124m   v1.20.1
node2.k8s    Ready    <none>                 120m   v1.20.1


03. Работа по задаче развертывания с помощью jenkins приложения в локальный kubernetes кластер

Клонируем к себе в репо на гитхаб

https://github.com/linuxacademy/cicd-pipeline-train-schedule-kubernetes


В Jenkins Нужно установить Plugin’ы:

  • “Docker Pipeline”
  • “Kubernetes Continuous Deploy”
  • “Publish Over SSH” (может и не нужен здесь)


Добавление Credentials:

Jenkins -> Add Credentials ->


Kind -> Kubernetes configuration (kubeconfig)

id: kubeconfig Description: Kubeconfig

Kubeconfig -> Enter directly ->

Вставляю содержимое файла

$ cat ~/.kube/config


Kubeconfig


Создание нового задания

Jenkins -> New Item

Name: train-schedule Type: Multibranch Pipeline


Branch Source -> GitHub

Repository HTTPS URL

https://github.com/webmak1/cicd-pipeline-train-schedule-kubernetes

validate.


Изменения в проекте для Deploy

У меня уже все сделано!


Делается следующим образом.

Добавляем в проект github

https://github.com/linuxacademy/cicd-pipeline-train-schedule-kubernetes/blob/example-solution/train-schedule-kube.yml


Заменяем Jenkinsfile и в нем docker-hub:

https://github.com/linuxacademy/cicd-pipeline-train-schedule-kubernetes/blob/example-solution/Jenkinsfile


На шаге развертывания ошибка.

ERROR: ERROR: Can't construct a java object for tag:yaml.org,2002:io.kubernetes.client.openapi.models.V1Service; exception=Class not found: io.kubernetes.client.openapi.models.V1Service
 in 'reader', line 1, column 1:
    kind: Service
    ^

hudson.remoting.ProxyException: Can't construct a java object for tag:yaml.org,2002:io.kubernetes.client.openapi.models.V1Service; exception=Class not found: io.kubernetes.client.openapi.models.V1Service
 in 'reader', line 1, column 1:
    kind: Service
    ^

	at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:335)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObjectNoCheck(BaseConstructor.java:229)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:219)
	at io.kubernetes.client.util.Yaml$CustomConstructor.constructObject(Yaml.java:337)
	at org.yaml.snakeyaml.constructor.BaseConstructor.constructDocument(BaseConstructor.java:173)
	at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:157)
	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:490)
	at org.yaml.snakeyaml.Yaml.loadAs(Yaml.java:456)
	at io.kubernetes.client.util.Yaml.loadAs(Yaml.java:224)
	at io.kubernetes.client.util.Yaml.modelMapper(Yaml.java:494)
	at io.kubernetes.client.util.Yaml.loadAll(Yaml.java:272)
	at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.apply(KubernetesClientWrapper.java:236)
	at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.doCall(DeploymentCommand.java:172)
	at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:124)
	at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:106)
	at hudson.FilePath.act(FilePath.java:1164)
	at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:68)
	at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:45)
	at com.microsoft.jenkins.azurecommons.command.CommandService.runCommand(CommandService.java:88)
	at com.microsoft.jenkins.azurecommons.command.CommandService.execute(CommandService.java:96)
	at com.microsoft.jenkins.azurecommons.command.CommandService.executeCommands(CommandService.java:75)
	at com.microsoft.jenkins.azurecommons.command.BaseCommandContext.executeCommands(BaseCommandContext.java:77)
	at com.microsoft.jenkins.kubernetes.KubernetesDeploy.perform(KubernetesDeploy.java:42)
	at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:54)
	at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:35)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: hudson.remoting.ProxyException: org.yaml.snakeyaml.error.YAMLException: Class not found: io.kubernetes.client.openapi.models.V1Service
	at org.yaml.snakeyaml.constructor.Constructor.getClassForNode(Constructor.java:664)
	at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.getConstructor(Constructor.java:322)
	at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:331)
	... 30 more
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

Could not update commit status, please check if your scan credentials belong to a member of the organization or a collaborator of the repository and repo:status scope is selected


GitHub has been notified of this commit’s build result

ERROR: Kubernetes deployment ended with HasError
Finished: FAILURE


Ничего не заработало с помощью этого Jenkins

Пишут, что нужно сделать DownGrade плагинов:

Jackson 2 API v2.10.0,

Kubernetes v1.21.3,

Kubernetes Client API v4.6.3-1,

Kubernetes Continuous Deploy v2.1.2,

Kubernetes Credentials v0.5.0


Возможно, что также нужно делать DownGrade:


Snakeyaml API to v1.26.2
GitHub Branch Source 2.7.1


Downgrade:
https://www.youtube.com/watch?v=d6BU8LBc9Ow


Скачать плагины:
https://plugins.jenkins.io/


Jenkins -> Plugins -> Advanced -> Upload


Запускаю руками

$ kubectl get nodes
NAME         STATUS   ROLES                  AGE   VERSION
master.k8s   Ready    control-plane,master   19h   v1.20.1
node1.k8s    Ready    <none>                 19h   v1.20.1
node2.k8s    Ready    <none>                 19h   v1.20.1


$ cat <<EOF | kubectl apply -f -
kind: Service
apiVersion: v1
metadata:
  name: train-schedule-service
spec:
  type: NodePort
  selector:
    app: train-schedule
  ports:
  - protocol: TCP
    port: 8080
    nodePort: 30001

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: train-schedule-deployment
  labels:
    app: train-schedule
spec:
  replicas: 2
  selector:
    matchLabels:
      app: train-schedule
  template:
    metadata:
      labels:
        app: train-schedule
    spec:
      containers:
      - name: train-schedule
        image: webmakaka/train-schedule
        ports:
        - containerPort: 8080
EOF


$ kubectl get pods
NAME                                         READY   STATUS    RESTARTS   AGE
train-schedule-deployment-67bfb5f9db-29cnj   1/1     Running   0          100s
train-schedule-deployment-67bfb5f9db-q8kvm   1/1     Running   0          100s


$ kubectl get svc
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
kubernetes               ClusterIP   10.96.0.1       <none>        443/TCP          19h
train-schedule-service   NodePort    10.105.61.230   <none>        8080:30001/TCP   73s


http://node1.k8s:30001


Kubeconfig


Удаляю созданные ресурсы

$ kubectl delete svc train-schedule-service
$ kubectl delete deployment train-schedule-deployment