k8s环境之cicd部署+远程触发

一、jenkins ci构建

def createVersion() {return new Date().format('yyyyMMddHHmmss') }pipeline {agent any environment {_version = createVersion()} parameters {gitParameter branchFilter: 'origin/(.*)', defaultValue: 'develop', name: 'BRANCH', type: 'PT_BRANCH'string defaultValue: 'mvn clean package -am -pl consumer', description: '打包命令', name: 'mvnArgs', trim: false} tools {maven 'MAVEN'jdk 'JDK'nodejs 'NODEJS'}stages {stage('pull') {steps {cleanWs()checkout([$class: 'GitSCM', branches: [[name: "${params.BRANCH}"]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'bedxxxxxxxxxxxxxxx591a2af54', url: "http://git.com/api.git"]]])}}stage('MVN') {steps {sh "${params.mvnArgs}"}}stage('docker-build') {steps {script{env.COMMIT= sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()sh """#!/bin/bashecho ${COMMIT}echo ${_version}docker login test.com --username 1xxxxxx1 --password xxxxxxxxxxxxxcd vota-api-consumer/docker build --build-arg consumer -t test.com/dev/consumer:${params.BRANCH}-${COMMIT}-${_version} .docker push test.com/dev/consumer:${params.BRANCH}-${COMMIT}-${_version}docker rmi test.com/dev/consumer:${params.BRANCH}-${COMMIT}-${_version}echo "${params.BRANCH}-${COMMIT}-${_version}" > tagidcurl --location --request POST --insecure \'https://cicd:1xxxxxxxxxeb56f47001e9fa50@cdi-apicom/job/svw-devconsumer/buildWithParameters' \-F 'token=11bxxxxxxxf47001e9fa50' \-F "tag=`cat tagid`" \-F 'data={}'"""}}} }}

二、cd构建—k8s部署的jenkins

def label = "slave-${UUID.randomUUID().toString()}"podTemplate(label: label, containers: [containerTemplate(name: 'kubectl', image: 'cnych/kubectl', command: 'cat', ttyEnabled: true) ], serviceAccount: 'jenkins', volumes: [hostPathVolume(mountPath: '/home/jenkins/.kube', hostPath: '/var/lib/container/jenkins/.kube'), ]) {node(label) {parameters {// string defaultValue: ' ', description: '请输入需要部署的consumer服务的image tag', name: 'image_tag', trim: falsestring defaultValue: 'tag', description: '镜像tag', name: "COMMIT", trim: false//string defaultValue: 'Version', description: 'tage', name: "Version", trim: false}stage('运行 Kubectl') {container('kubectl') {// echo "${image_tag}"// echo "${service}-${COMMIT}-${Version}"sh "kubectl set image deployment/consumer consumer=test.com/dev/consumer:${tag} -n dev"}}} }

三、k8s部署jenkins

apiVersion: v1 kind: ServiceAccount metadata:name: jenkinsnamespace: dev---apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata:name: jenkins rules:- apiGroups: ["extensions", "apps"]resources: ["deployments"]verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]- apiGroups: [""]resources: ["services"]verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]- apiGroups: [""]resources: ["pods"]verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]resources: ["pods/exec"]verbs: ["create","delete","get","list","patch","update","watch"]- apiGroups: [""]resources: ["pods/log"]verbs: ["get","list","watch"]- apiGroups: [""]resources: ["secrets"]verbs: ["get"]- apiGroups: [""]resources: ["persistentvolume", "persistentvolumeclaims"]verbs: ["update", "get", "list", "patch", "watch"]- apiGroups: [""]resources: ["configmaps"]verbs: ["create", "update", "get", "list", "patch", "watch"]--- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata:name: jenkinsnamespace: dev roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: jenkins subjects:- kind: ServiceAccountname: jenkinsnamespace: devjenkins.deploy.yamlapiVersion: v1 kind: Service metadata:name: jenkinsnamespace: devlabels:app: jenkins spec:selector:app: jenkinstype: NodePortports:- name: webport: 8080targetPort: webnodePort: 32080- name: agentport: 50000targetPort: agent --- apiVersion: apps/v1 kind: Deployment metadata:name: jenkinsnamespace: dev spec:selector:matchLabels:app: jenkinstemplate:metadata:labels:app: jenkinsspec:nodeSelector:cm: test #给节点打标签 固定jenkinspod----因为会指定config认证文件，如果pod漂移就需要所有节点都已添加认证文件了terminationGracePeriodSeconds: 10serviceAccount: jenkinscontainers:# affinity:# nodeAffinity:# requiredDuringSchedulingIgnoredDuringExecution: # 硬策略# nodeSelectorTerms:# - matchExpressions:# - key: cm# operator: In# values:# - test- name: jenkins#image: jenkins:2.60.3image: jenkins/jenkins:ltsimagePullPolicy: IfNotPresentports:- containerPort: 8080name: web- containerPort: 50000name: agentresources:limits:cpu: 1000mmemory: 1Girequests:cpu: 500mmemory: 512MilivenessProbe:httpGet:path: /loginport: 8080initialDelaySeconds: 60timeoutSeconds: 5failureThreshold: 12readinessProbe:httpGet:path: /loginport: 8080initialDelaySeconds: 60timeoutSeconds: 5failureThreshold: 12volumeMounts:- name: jenkinshomemountPath: /var/jenkins_homesecurityContext:fsGroup: 1000runAsUser: 0volumes:- name: jenkinshomehostPath:path: /var/lib/container/jenkins #需要在同级目录下创建添加.kube/config文件 pipeline要调用kubectl命令type: Directory# persistentVolumeClaim:# claimName: ota-jenkins-cd-pvc

四、jenkins远程触发配置

由于带了map的选择打包构建，会有参数来选择service是发布哪一个

但是如果是远程构建，远程jenkins也采用map的形式，只能通过传参来触发构建

#需要第一个jenkins安装插件Parameterized Remote Trigger

原理：jenkins接口调用
1.在cdjenkins新建cicd用户

2.开启全局安全配置


3.获取CICD用户远程api-token

4.在需要被触发的任务中选择触发远程构建,并写入身份验证令牌

5.触发
#https://用户:api-token@jenkins地址/jenkins/job/任务名称/build?token=身份验证令牌

curl -X POST https://cicd:1131xxxxxxx71b84d466c06a8534b2f@jenkins_url/jenkins/job/${JOB_NAME}/build?token=bml5b3VsYWRxZGFkYXNkYWRhcWR4QEAjQCMK

带参数触发 echo "${params.BRANCH}-${COMMIT}-${_version}" > tagidcurl --location --request POST --insecure \'https://cicd:11be7bac2xxxxxxxxxx5eb56f47001e9fa50@cdi-api-gp-ota-dev.mos.csvw.com/job/svw-dev-cd-viov-security/buildWithParameters' \-F 'token=11be7bxxxxxxxxxxxxxx15eb56f47001e9fa50' \-F "tag=`cat tagid`" \-F 'data={}'

五、踩坑历程

1.cdjenkins 构建报错

解决：

1.在jenkins home path 也就是/var/lib/container/jenkins同级目录下创建添加.kube/config文件

2.通过nodeselector给节点打标签将jenkins pod 定死在一个认证的可执行kubectl的节点

解决：因为version参数在ci的jenkins是个变量，cd这边不能通过远程触发传参的方式获取值

将镜像的tag写死在文件里，echo 进tagid的文件在通过cat获取值，写死成常量即可远程传参获取

具体在ci的pipeline中体现

总结

以上是生活随笔为你收集整理的k8s环境之cicd部署+远程触发的全部内容，希望文章能够帮你解决所遇到的问题。

如果觉得生活随笔网站内容还不错，欢迎将生活随笔推荐给好友。