[kubernetes]-kubernetes部署jenkins编译发布pipeline项目

╰半夏微凉° 2022-12-02 15:23 214阅读 0赞

### 1 创建jenkins-master镜像 ###

资源下载地址：

链接: https://pan.baidu.com/s/1WUk8CTVkKYPGNz\_ufDKtPw 密码: 1n94

> 如果资源有问题 可以直接下官方镜像 然后从里面scp出来

需要如下文件 jdk安装包 tomcat 里面包含需要的jenkins的war包 名称为

1 ROOT.war

2 Dockerfile

3 settings.xml(如果不需要在master上编译的话 不要也没关系)

4 .ssh 文件 用于下载ssh 的git代码 以及config文件 用于避免下载代码的时候交互yea/no 否则可能会卡在git clone

> 这里tomcat里的war包 记得重命名 否则如果是jenkins.war打好镜像之后 使用slave编译的时候会提示找不到标签为xxx的slave agent，看了pod报错是找不到api 之类的。url 直接是/的 不是/jenkins，不太好改。

![format_png][]

Dockerfile

FROM centos:centos7.4.1708
    MAINTAINER PDABC Enterprise Container Images <jiaminxu@hz-health.cn>
    
    ADD jdk-8u161-linux-x64.tar.gz /usr/local/java/
    COPY tomcat /usr/local/tomcat
    COPY .ssh /root/.ssh
    RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
      && yum -y install git subversion sshpassi maven \
      && rm -rf /var/cache/yum
    
    COPY settings.xml /etc/maven/settings.xml
    WORKDIR /root
    ENV JAVA_HOME /usr/local/java/jdk1.8.0_161
    ENV JRE_HOME ${JAVA_HOME}/jre
    ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/lib
    ENV PATH ${JAVA_HOME}/bin:$PATH
    
    
    #公开端口
    EXPOSE 8080 5000
    
    USER root
    #设置启动命令
    ENTRYPOINT ["/usr/local/tomcat/bin/catalina.sh","run"]

docker build -t registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master01 .
    docker push registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master01

为什么创建自己的master镜像 原因如下

[root@op-k8s-m001 jenkins]# kubectl logs -f  jenkins-6db584c4cd-lfl9g -n devops
    VM settings:
        Max. Heap Size (Estimated): 247.50M
        Ergonomics Machine Class: server
        Using VM: OpenJDK 64-Bit Server VM
    
    Running from: /usr/share/jenkins/jenkins.war
    webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
    2020-08-31 06:37:21.552+0000 [id=1]	INFO	org.eclipse.jetty.util.log.Log#initialized: Logging initialized @646ms to org.eclipse.jetty.util.log.JavaUtilLog
    2020-08-31 06:37:21.832+0000 [id=1]	INFO	winstone.Logger#logInternal: Beginning extraction from war file
    2020-08-31 06:37:23.518+0000 [id=1]	WARNING	o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
    2020-08-31 06:37:23.571+0000 [id=1]	INFO	org.eclipse.jetty.server.Server#doStart: jetty-9.4.z-SNAPSHOT; built: 2019-05-02T00:04:53.875Z; git: e1bc35120a6617ee3df052294e433f3a25ce7097; jvm 1.8.0_222-b10
    2020-08-31 06:37:24.132+0000 [id=1]	INFO	o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
    2020-08-31 06:37:24.222+0000 [id=1]	INFO	o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0
    2020-08-31 06:37:24.223+0000 [id=1]	INFO	o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults
    2020-08-31 06:37:24.225+0000 [id=1]	INFO	o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 600000ms
    Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
    2020-08-31 06:37:25.231+0000 [id=1]	INFO	o.e.j.s.handler.ContextHandler#doStart: Started w.@26be6ca7{Jenkins v2.199,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
    2020-08-31 06:37:25.247+0000 [id=1]	INFO	o.e.j.server.AbstractConnector#doStart: Started ServerConnector@19932c16{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
    2020-08-31 06:37:25.247+0000 [id=1]	INFO	org.eclipse.jetty.server.Server#doStart: Started @4342ms
    2020-08-31 06:37:25.248+0000 [id=20]	INFO	winstone.Logger#logInternal: Winstone Servlet Engine v4.0 running: controlPort=disabled
    2020-08-31 06:37:27.660+0000 [id=25]	INFO	jenkins.InitReactorRunner$1#onAttained: Started initialization
    2020-08-31 06:37:29.064+0000 [id=26]	WARNING	hudson.ClassicPluginStrategy#createPluginWrapper: encountered /var/jenkins_home/plugins/changelog-environment.hpi under a nonstandard name; expected changelog-environment.jpi
    2020-08-31 06:37:29.071+0000 [id=26]	INFO	jenkins.InitReactorRunner$1#onAttained: Listed all plugins
    2020-08-31 06:37:30.038+0000 [id=26]	INFO	com.sic.plugins.kpp.KPPPlugin#start: starting kpp plugin
    2020-08-31 06:37:30.065+0000 [id=26]	INFO	o.j.h.p.m.M2ReleaseBuildWrapper$DescriptorImpl#<clinit>: Using new style Permission with PermissionScope
    2020-08-31 06:37:30.128+0000 [id=26]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin Pipeline Maven Integration Plugin v3.8.2 (pipeline-maven)
    java.io.IOException: Pipeline Maven Integration Plugin version 3.8.2 failed to load.
     - HTML Publisher plugin version 1.14 is older than required. To fix, install version 1.16 or later.
     - Jenkins JaCoCo plugin version 2.2.1 is older than required. To fix, install version 3.0.3 or later.
    	at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:922)
    	at hudson.PluginManager$2$1$1.run(PluginManager.java:546)
    	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
    	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
    	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:748)
    2020-08-31 06:37:30.142+0000 [id=25]	INFO	ruby.RubyRuntimePlugin#start: Injecting JRuby into XStream
    2020-08-31 06:37:55.440+0000 [id=26]	INFO	jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
    2020-08-31 06:37:55.730+0000 [id=26]	INFO	jenkins.InitReactorRunner$1#onAttained: Started all plugins
    2020-08-31 06:37:57.349+0000 [id=26]	WARNING	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate Key[type=jenkins.telemetry.Correlator, annotation=[none]]; skipping this component
    com.google.inject.ProvisionException: Unable to provision, see the following errors:
    
    1) Tried proxying jenkins.telemetry.Correlator to support a circular dependency, but it is not an interface.
    
    1 error
    	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:52)
    	at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145)
    	at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:439)
    	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
    	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
    	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
    	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
    	at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:401)
    	at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:392)
    	at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:344)
    	at hudson.ExtensionList.load(ExtensionList.java:381)
    	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
    	at hudson.ExtensionList.getComponents(ExtensionList.java:183)
    	at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:193)
    	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
    	at hudson.ExtensionList.iterator(ExtensionList.java:172)
    	at hudson.model.User.allocateDefaultPropertyInstancesAsNeeded(User.java:206)
    	at hudson.model.User.load(User.java:195)
    	at hudson.model.User.<init>(User.java:188)
    	at hudson.model.User.getOrCreateById(User.java:509)
    	at hudson.model.User.getById(User.java:601)
    	at hudson.model.User.get(User.java:586)
    	at hudson.plugins.jobConfigHistory.PluginUtils.getSystemHistoryDao(PluginUtils.java:132)
    	at hudson.plugins.jobConfigHistory.JobConfigHistorySaveableListener.getHistoryDao(JobConfigHistorySaveableListener.java:92)
    	at hudson.plugins.jobConfigHistory.JobConfigHistorySaveableListener.onChange(JobConfigHistorySaveableListener.java:56)
    	at hudson.model.listeners.SaveableListener.fireOnChange(SaveableListener.java:81)
    	at hudson.model.Descriptor.save(Descriptor.java:877)
    	at jenkins.telemetry.Correlator.<init>(Correlator.java:52)
    	at jenkins.telemetry.Correlator$$FastClassByGuice$$5678e8d6.newInstance(<generated>)
    	at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
    	at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:61)
    	at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:105)
    	at com.google.inject.internal.ConstructorInjector.access$000(ConstructorInjector.java:32)
    	at com.google.inject.internal.ConstructorInjector$1.call(ConstructorInjector.java:89)
    	at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115)
    	at hudson.ExtensionFinder$GuiceFinder$SezpozModule.onProvision(ExtensionFinder.java:567)
    	at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126)
    	at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68)
    	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:87)
    	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)
    	at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
    	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1103)
    	at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
    	at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:145)
    	at hudson.ExtensionFinder$GuiceFinder$FaultTolerantScope$1.get(ExtensionFinder.java:439)
    	at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
    	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
    	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
    	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
    	at hudson.ExtensionFinder$GuiceFinder._find(ExtensionFinder.java:401)
    	at hudson.ExtensionFinder$GuiceFinder.find(ExtensionFinder.java:392)
    	at hudson.ClassicPluginStrategy.findComponents(ClassicPluginStrategy.java:344)
    	at hudson.ExtensionList.load(ExtensionList.java:381)
    	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
    	at hudson.ExtensionList.getComponents(ExtensionList.java:183)
    	at hudson.DescriptorExtensionList.load(DescriptorExtensionList.java:193)
    	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:317)
    	at hudson.ExtensionList.iterator(ExtensionList.java:172)
    	at jenkins.branch.CustomOrganizationFolderDescriptor.doAddSpecificDescriptors(CustomOrganizationFolderDescriptor.java:166)
    	at jenkins.branch.CustomOrganizationFolderDescriptor.addSpecificDescriptors(CustomOrganizationFolderDescriptor.java:145)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
    	at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
    	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
    	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1124)
    	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
    	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:748)
    2020-08-31 06:37:58.522+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component hudson.plugins.build_timeout.operations.AbortAndRestartOperation$DescriptorImpl; skipping
    2020-08-31 06:38:00.026+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component org.biouno.unochoice.model.ScriptlerScript$DescriptorImpl; skipping
    2020-08-31 06:38:00.733+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component hudson.plugins.build_timeout.operations.AbortAndRestartOperation$DescriptorImpl; skipping
    2020-08-31 06:38:00.735+0000 [id=26]	INFO	h.ExtensionFinder$GuiceFinder$FaultTolerantScope$1#error: Failed to instantiate optional component org.biouno.unochoice.model.ScriptlerScript$DescriptorImpl; skipping
    2020-08-31 06:38:00.819+0000 [id=25]	INFO	jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
    2020-08-31 06:38:00.943+0000 [id=25]	INFO	jenkins.InitReactorRunner$1#onAttained: Loaded all jobs
    2020-08-31 06:38:01.127+0000 [id=42]	INFO	hudson.model.AsyncPeriodicWork$1#run: Started Download metadata
    2020-08-31 06:38:01.134+0000 [id=42]	INFO	hudson.util.Retrier#start: Attempt #1 to do the action check updates server
    2020-08-31 06:38:02.449+0000 [id=26]	WARNING	o.j.r.u.AnonymousClassWarnings#warn: Attempt to (de-)serialize anonymous class com.gitee.jenkins.gitee.api.impl.GiteeV5ClientBuilder$1 in file:/var/jenkins_home/plugins/gitee/WEB-INF/lib/gitee-1.1.3.jar; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
    2020-08-31 06:38:02.747+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#prepareRefresh: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@c553191: display name [Root WebApplicationContext]; startup date [Mon Aug 31 14:38:02 CST 2020]; root of context hierarchy
    2020-08-31 06:38:02.747+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#obtainFreshBeanFactory: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@c553191]: org.springframework.beans.factory.support.DefaultListableBeanFactory@263fd32b
    2020-08-31 06:38:02.761+0000 [id=25]	INFO	o.s.b.f.s.DefaultListableBeanFactory#preInstantiateSingletons: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@263fd32b: defining beans [authenticationManager]; root of factory hierarchy
    2020-08-31 06:38:03.224+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#prepareRefresh: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@1c077129: display name [Root WebApplicationContext]; startup date [Mon Aug 31 14:38:03 CST 2020]; root of context hierarchy
    2020-08-31 06:38:03.224+0000 [id=25]	INFO	o.s.c.s.AbstractApplicationContext#obtainFreshBeanFactory: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@1c077129]: org.springframework.beans.factory.support.DefaultListableBeanFactory@214772f9
    2020-08-31 06:38:03.225+0000 [id=25]	INFO	o.s.b.f.s.DefaultListableBeanFactory#preInstantiateSingletons: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@214772f9: defining beans [filter,legacy]; root of factory hierarchy
    2020-08-31 06:38:03.958+0000 [id=25]	INFO	jenkins.install.SetupWizard#init:
    
    *************************************************************
    *************************************************************
    *************************************************************
    
    Jenkins initial setup is required. An admin user has been created and a password generated.
    Please use the following password to proceed to installation:
    
    73281c7d200643cc8256a225f7c49d9f
    
    This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
    
    *************************************************************
    *************************************************************
    *************************************************************
    
    ^C
    [root@op-k8s-m001 jenkins]#
    [root@op-k8s-m001 jenkins]#
    [root@op-k8s-m001 jenkins]# kubectl describe pod jenkins-6db584c4cd-lfl9g -n devops
    Name:               jenkins-6db584c4cd-lfl9g
    Namespace:          devops
    Priority:           0
    PriorityClassName:  <none>
    Node:               op-k8s-n002/192.168.1.205
    Start Time:         Mon, 31 Aug 2020 14:16:21 +0800
    Labels:             app=jenkins
                        pod-template-hash=6db584c4cd
    Annotations:        <none>
    Status:             Running
    IP:                 10.244.2.208
    Controlled By:      ReplicaSet/jenkins-6db584c4cd
    Containers:
      jenkins:
        Container ID:   docker://54b927fc35395df798bf5983abdcd3bb4db337b37497d25910c1cc97489e7174
        Image:          registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199
        Image ID:       docker-pullable://registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins@sha256:7ab00ec2a9601b9261e50ed294b75f9a0ccfa2750070bd37b31ec80be3ece887
        Ports:          8080/TCP, 50000/TCP
        Host Ports:     0/TCP, 0/TCP
        State:          Running
          Started:      Mon, 31 Aug 2020 14:37:20 +0800
        Last State:     Terminated
          Reason:       Error
          Exit Code:    143
          Started:      Mon, 31 Aug 2020 14:34:21 +0800
          Finished:     Mon, 31 Aug 2020 14:37:20 +0800
        Ready:          False
        Restart Count:  7
        Limits:
          cpu:     1
          memory:  1Gi
        Requests:
          cpu:      500m
          memory:   1Gi
        Liveness:   http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
        Readiness:  http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
        Environment:
          JAVA_OPTS:  -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
        Mounts:
          /var/jenkins_home/plugins from pluginshome (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from jenkins-sa-token-p4rdt (ro)
    Conditions:
      Type              Status
      Initialized       True
      Ready             False
      ContainersReady   False
      PodScheduled      True
    Volumes:
      test-jenkinshome:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  test-pvc
        ReadOnly:   false
      m2home:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  m2-pvc
        ReadOnly:   false
      pluginshome:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  jenkins-plugins-pvc
        ReadOnly:   false
      jenkins-sa-token-p4rdt:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  jenkins-sa-token-p4rdt
        Optional:    false
    QoS Class:       Burstable
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                     node.kubernetes.io/unreachable:NoExecute for 300s
    Events:
      Type     Reason     Age                  From                  Message
      ----     ------     ----                 ----                  -------
      Normal   Scheduled  22m                  default-scheduler     Successfully assigned devops/jenkins-6db584c4cd-lfl9g to op-k8s-n002
      Normal   Pulled     22m                  kubelet, op-k8s-n002  Container image "registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199" already present on machine
      Normal   Created    22m                  kubelet, op-k8s-n002  Created container jenkins
      Normal   Started    22m                  kubelet, op-k8s-n002  Started container jenkins
      Warning  Unhealthy  19m (x11 over 21m)   kubelet, op-k8s-n002  Liveness probe failed: HTTP probe failed with statuscode: 503
      Warning  Unhealthy  2m8s (x79 over 21m)  kubelet, op-k8s-n002  Readiness probe failed: HTTP probe failed with statuscode: 503
    [root@op-k8s-m001 jenkins]# kubectl get pods -n devops -o wide -w
    NAME                       READY   STATUS             RESTARTS   AGE   IP             NODE          NOMINATED NODE   READINESS GATES
    jenkins-6db584c4cd-lfl9g   0/1     CrashLoopBackOff   7          24m   10.244.2.208   op-k8s-n002   <none>           <none>
    
    
    
    ^C[root@op-k8s-m001 jenkins]# kubectl describe pod jenkins-6db584c4cd-lfl9g -n devops
    Name:               jenkins-6db584c4cd-lfl9g
    Namespace:          devops
    Priority:           0
    PriorityClassName:  <none>
    Node:               op-k8s-n002/192.168.1.205
    Start Time:         Mon, 31 Aug 2020 14:16:21 +0800
    Labels:             app=jenkins
                        pod-template-hash=6db584c4cd
    Annotations:        <none>
    Status:             Running
    IP:                 10.244.2.208
    Controlled By:      ReplicaSet/jenkins-6db584c4cd
    Containers:
      jenkins:
        Container ID:   docker://54b927fc35395df798bf5983abdcd3bb4db337b37497d25910c1cc97489e7174
        Image:          registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199
        Image ID:       docker-pullable://registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins@sha256:7ab00ec2a9601b9261e50ed294b75f9a0ccfa2750070bd37b31ec80be3ece887
        Ports:          8080/TCP, 50000/TCP
        Host Ports:     0/TCP, 0/TCP
        State:          Waiting
          Reason:       CrashLoopBackOff
        Last State:     Terminated
          Reason:       Error
          Exit Code:    143
          Started:      Mon, 31 Aug 2020 14:37:20 +0800
          Finished:     Mon, 31 Aug 2020 14:40:20 +0800
        Ready:          False
        Restart Count:  7
        Limits:
          cpu:     1
          memory:  1Gi
        Requests:
          cpu:      500m
          memory:   1Gi
        Liveness:   http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
        Readiness:  http-get http://:8080/login delay=60s timeout=5s period=10s #success=1 #failure=12
        Environment:
          JAVA_OPTS:  -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
        Mounts:
          /var/jenkins_home/plugins from pluginshome (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from jenkins-sa-token-p4rdt (ro)
    Conditions:
      Type              Status
      Initialized       True
      Ready             False
      ContainersReady   False
      PodScheduled      True
    Volumes:
      test-jenkinshome:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  test-pvc
        ReadOnly:   false
      m2home:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  m2-pvc
        ReadOnly:   false
      pluginshome:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  jenkins-plugins-pvc
        ReadOnly:   false
      jenkins-sa-token-p4rdt:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  jenkins-sa-token-p4rdt
        Optional:    false
    QoS Class:       Burstable
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                     node.kubernetes.io/unreachable:NoExecute for 300s
    Events:
      Type     Reason     Age                   From                  Message
      ----     ------     ----                  ----                  -------
      Normal   Scheduled  24m                   default-scheduler     Successfully assigned devops/jenkins-6db584c4cd-lfl9g to op-k8s-n002
      Normal   Pulled     24m                   kubelet, op-k8s-n002  Container image "registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199" already present on machine
      Normal   Created    24m                   kubelet, op-k8s-n002  Created container jenkins
      Normal   Started    24m                   kubelet, op-k8s-n002  Started container jenkins
      Warning  Unhealthy  21m (x11 over 23m)    kubelet, op-k8s-n002  Liveness probe failed: HTTP probe failed with statuscode: 503
      Warning  Unhealthy  4m30s (x79 over 23m)  kubelet, op-k8s-n002  Readiness probe failed: HTTP probe failed with statuscode: 503
    [root@op-k8s-m001 jenkins]#

原来官方镜像jenkins/jenkins:2.199启动时间参考 每次都会重启至少一次 启动时间5m以上，有时候甚至起不来一直卡在503

![format_png 1][]

现在新镜像时间参考 大小的话原先300M不到 我新作的500M不到。大小有差别 但是个人比较喜欢用centos

![format_png 2][]

健康检查通过比较快 不过我后来是注释掉了readiness和liveness

![format_png 3][]

2 创建pvc rbac deploy的yaml文件如下

首先将需要持久化存储 共享的文件通过nfs 挂载出来 我这里是选择192.168.1.171

> maven 仓库 一些jenkins的文件夹(如插件 工作路径等，不然我每次都要重新下插件 下载代码比较耗时也比较麻烦) 我拿/opt/workspace是来测试

![format_png 4][]

jenkins-pvc.yaml

---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: jenkins-pv
      namespace: devops
    spec:
      capacity:
        storage: 200Gi
      accessModes:
      - ReadWriteMany
      persistentVolumeReclaimPolicy: Delete
      nfs:
        server: 192.168.1.171
        path: /root/.jenkins
    
    
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: jenkins-pvc
      namespace: devops
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 200Gi
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: test-pv
      namespace: devops
    spec:
      capacity:
        storage: 50Gi
      accessModes:
      - ReadWriteMany
      persistentVolumeReclaimPolicy: Delete
      nfs:
        server: 192.168.1.171
        path: /opt/workspace
    
    
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: test-pvc
      namespace: devops
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 50Gi
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: m2-pv
      namespace: devops
    spec:
      capacity:
        storage: 200Gi
      accessModes:
      - ReadWriteMany
      persistentVolumeReclaimPolicy: Delete
      nfs:
        server: 192.168.1.171
        path: /srv/m2
    
    
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: m2-pvc
      namespace: devops
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 200Gi
    
    
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: jenkins-plugins-pv
      namespace: devops
    spec:
      capacity:
        storage: 200Gi
      accessModes:
      - ReadWriteMany
      persistentVolumeReclaimPolicy: Delete
      nfs:
        server: 192.168.1.171
        path: /root/.jenkins/plugins
    
    
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: jenkins-plugins-pvc
      namespace: devops
    spec:
      accessModes:
      - ReadWriteMany
      resources:
        requests:
          storage: 200Gi

jenkins-rbac.yaml

apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: jenkins-sa
      namespace: devops
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRole
    metadata:
      name: jenkins-cr
    rules:
      - apiGroups: ["extensions", "apps"]
        resources: ["deployments"]
        verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
      - apiGroups: [""]
        resources: ["services"]
        verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["create","delete","get","list","patch","update","watch"]
      - apiGroups: [""]
        resources: ["pods/exec"]
        verbs: ["create","delete","get","list","patch","update","watch"]
      - apiGroups: [""]
        resources: ["pods/log"]
        verbs: ["get","list","watch"]
      - apiGroups: [""]
        resources: ["secrets"]
        verbs: ["get"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: jenkins-crd
    roleRef:
      kind: ClusterRole
      name: jenkins-cr
      apiGroup: rbac.authorization.k8s.io
    subjects:
    - kind: ServiceAccount
      name: jenkins-sa
      namespace: devops

test-jenkins-deploy.yaml

---
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: jenkins
      namespace: devops
    spec:
      template:
        metadata:
          labels:
            app: jenkins
        spec:
          terminationGracePeriodSeconds: 10
          serviceAccount: jenkins-sa
          containers:
          - name: jenkins
            image:  registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master0406
            #image: registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:2.199base01
            env:
            - name: JAVA_OPTS
              value: -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
            ports:
            - containerPort: 8080
              name: web
              protocol: TCP
            - containerPort: 50000
              name: agent
              protocol: TCP
            resources:
              limits:
                cpu: 2000m
                memory: 2Gi
              requests:
                cpu: 1000m
                memory: 1Gi
    # livenessProbe:
    # httpGet:
    # path: /jenkins/login
    # port: 8080
    # initialDelaySeconds: 60
    # timeoutSeconds: 5
    # failureThreshold: 12
    # readinessProbe:
    # httpGet:
    # path: /jenkins/login
    # port: 8080
    # initialDelaySeconds: 60
    # timeoutSeconds: 5
    # failureThreshold: 12
            volumeMounts:
            - name: test-jenkinshome
              mountPath: /root/.jenkins/workspace
            - name: pluginshome
              mountPath: /root/.jenkins/plugins
            - name: m2home
              mountPath: /srv/m2
          securityContext:
            fsGroup: 1000
          volumes:
          - name: test-jenkinshome
            persistentVolumeClaim:
              claimName: test-pvc
          - name: m2home
            persistentVolumeClaim:
              claimName: m2-pvc
          - name: pluginshome
            persistentVolumeClaim:
              claimName: jenkins-plugins-pvc
    
          imagePullSecrets:
          - name: aliyun-ns-devops
    
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: jenkins
      namespace: devops
      labels:
        app: jenkins
    spec:
      selector:
        app: jenkins
      type: NodePort
      ports:
      - name: web
        port: 8080
        targetPort: web
        nodePort: 30002
      - name: agent
        port: 50000 # 这个要和代理指定端口一致
        targetPort: agent

kubectl apply -f  jenkins-pvc.yaml
    kubectl apply -f  jenkins-rbac.yaml
    kubectl apply -f  test-jenkins-deploy.yaml

### 3 调整jenkins-master镜像 ###

jenkins 启动慢 经常卡在jenkins正在启动 状态为503 主要需要修改2个文件/root/.jenkins/hudson.model.UpdateCenter.xml和/root/.jenkins/updates/default.json

修改这两个文件

cat > /root/.jenkins/hudson.model.UpdateCenter.xml <<EOF
    <?xml version='1.1' encoding='UTF-8'?>
    <sites>
      <site>
        <id>default</id>
        <url>http://mirror.xmission.com/jenkins/updates/update-center.json</url>
      </site>
    </sites>
    EOF
    
    sed -i 's/www.google.com/www.baidu.com/g'  /root/.jenkins/updates/default.json

##### 登陆jenkins配置基础项 #####

###  ###

密码通过 kubectl exec $pod\_name -n devops 之后 cat /root/.jenkins/secrets/initialAdminPassword 查看

![format_png 5][]

进入系统管理 > 系统配置 > 新增一个云(需要先装上kubernetes插件)

![format_png 6][]

![format_png 7][]

![format_png 8][]

![format_png 9][]

![format_png 10][]

上面配置的config.xml如下

<?xml version='1.1' encoding='UTF-8'?>
    <hudson>
      <disabledAdministrativeMonitors/>
      <version>2.199</version>
      <installStateName>RUNNING</installStateName>
      <numExecutors>2</numExecutors>
      <mode>NORMAL</mode>
      <useSecurity>true</useSecurity>
      <authorizationStrategy class="hudson.security.FullControlOnceLoggedInAuthorizationStrategy">
        <denyAnonymousReadAccess>true</denyAnonymousReadAccess>
      </authorizationStrategy>
      <securityRealm class="hudson.security.HudsonPrivateSecurityRealm">
        <disableSignup>true</disableSignup>
        <enableCaptcha>false</enableCaptcha>
      </securityRealm>
      <disableRememberMe>false</disableRememberMe>
      <projectNamingStrategy class="jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy"/>
      <workspaceDir>${JENKINS_HOME}/workspace/${ITEM_FULL_NAME}</workspaceDir>
      <buildsDir>${ITEM_ROOTDIR}/builds</buildsDir>
      <markupFormatter class="hudson.markup.EscapedMarkupFormatter"/>
      <jdks/>
      <viewsTabBar class="hudson.views.DefaultViewsTabBar"/>
      <myViewsTabBar class="hudson.views.DefaultMyViewsTabBar"/>
      <clouds>
        <org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud plugin="kubernetes@1.21.1">
          <name>kubernetes</name>
          <defaultsProviderTemplate></defaultsProviderTemplate>
          <templates>
            <org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
              <inheritFrom></inheritFrom>
              <name>jenkins-slave</name>
              <namespace>devops</namespace>
              <privileged>false</privileged>
              <capOnlyOnAlivePods>false</capOnlyOnAlivePods>
              <alwaysPullImage>false</alwaysPullImage>
              <instanceCap>2147483647</instanceCap>
              <slaveConnectTimeout>100</slaveConnectTimeout>
              <idleMinutes>0</idleMinutes>
              <activeDeadlineSeconds>0</activeDeadlineSeconds>
              <label>jenkins-jnlp</label>
              <serviceAccount>jenkins-sa</serviceAccount>
              <nodeSelector></nodeSelector>
              <nodeUsageMode>EXCLUSIVE</nodeUsageMode>
              <hostNetwork>false</hostNetwork>
              <workspaceVolume class="org.csanchez.jenkins.plugins.kubernetes.volumes.workspace.EmptyDirWorkspaceVolume">
                <memory>false</memory>
              </workspaceVolume>
              <volumes>
                <org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
                  <mountPath>/var/run/docker.sock</mountPath>
                  <hostPath>/var/run/docker.sock</hostPath>
                </org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
                <org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
                  <mountPath>/root/.kube</mountPath>
                  <hostPath>/root/.kube</hostPath>
                </org.csanchez.jenkins.plugins.kubernetes.volumes.HostPathVolume>
                <org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
                  <mountPath>/root/.jenkins/workspace</mountPath>
                  <serverAddress>192.168.1.171</serverAddress>
                  <serverPath>/opt/workspace</serverPath>
                  <readOnly>false</readOnly>
                </org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
                <org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
                  <mountPath>/usr/share/maven/conf</mountPath>
                  <serverAddress>192.168.1.171</serverAddress>
                  <serverPath>/usr/share/maven/conf</serverPath>
                  <readOnly>false</readOnly>
                </org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
                <org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
                  <mountPath>/srv/m2</mountPath>
                  <serverAddress>192.168.1.171</serverAddress>
                  <serverPath>/srv/m2</serverPath>
                  <readOnly>false</readOnly>
                </org.csanchez.jenkins.plugins.kubernetes.volumes.NfsVolume>
              </volumes>
              <containers>
                <org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>
                  <name>jnlp</name>
                  <image>registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:jnlp6.09</image>
                  <privileged>false</privileged>
                  <alwaysPullImage>false</alwaysPullImage>
                  <workingDir>/root/.jenkins</workingDir>
                  <command></command>
                  <args></args>
                  <ttyEnabled>true</ttyEnabled>
                  <resourceRequestCpu></resourceRequestCpu>
                  <resourceRequestMemory></resourceRequestMemory>
                  <resourceLimitCpu></resourceLimitCpu>
                  <resourceLimitMemory></resourceLimitMemory>
                  <envVars/>
                  <ports/>
                  <livenessProbe>
                    <execArgs></execArgs>
                    <timeoutSeconds>0</timeoutSeconds>
                    <initialDelaySeconds>0</initialDelaySeconds>
                    <failureThreshold>0</failureThreshold>
                    <periodSeconds>0</periodSeconds>
                    <successThreshold>0</successThreshold>
                  </livenessProbe>
                </org.csanchez.jenkins.plugins.kubernetes.ContainerTemplate>
              </containers>
              <envVars/>
              <annotations/>
              <imagePullSecrets>
                <org.csanchez.jenkins.plugins.kubernetes.PodImagePullSecret>
                  <name>aliyun-ns-devops</name>
                </org.csanchez.jenkins.plugins.kubernetes.PodImagePullSecret>
              </imagePullSecrets>
              <nodeProperties/>
              <yamlMergeStrategy class="org.csanchez.jenkins.plugins.kubernetes.pod.yaml.Overrides"/>
              <showRawYaml>true</showRawYaml>
              <podRetention class="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Default"/>
            </org.csanchez.jenkins.plugins.kubernetes.PodTemplate>
          </templates>
          <serverUrl>https://kubernetes.default.svc.cluster.local</serverUrl>
          <skipTlsVerify>false</skipTlsVerify>
          <addMasterProxyEnvVars>false</addMasterProxyEnvVars>
          <capOnlyOnAlivePods>false</capOnlyOnAlivePods>
          <namespace>devops</namespace>
          <directConnection>false</directConnection>
          <jenkinsUrl>http://jenkins.devops.svc.cluster.local:8080</jenkinsUrl>
          <containerCap>10</containerCap>
          <retentionTimeout>5</retentionTimeout>
          <connectTimeout>5</connectTimeout>
          <readTimeout>15</readTimeout>
          <podLabels>
            <org.csanchez.jenkins.plugins.kubernetes.PodLabel>
              <key>jenkins</key>
              <value>slave</value>
            </org.csanchez.jenkins.plugins.kubernetes.PodLabel>
          </podLabels>
          <usageRestricted>false</usageRestricted>
          <maxRequestsPerHost>32</maxRequestsPerHost>
          <waitForPodSec>600</waitForPodSec>
          <podRetention class="org.csanchez.jenkins.plugins.kubernetes.pod.retention.Never"/>
        </org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud>
      </clouds>
      <quietPeriod>5</quietPeriod>
      <scmCheckoutRetryCount>0</scmCheckoutRetryCount>
      <views>
        <hudson.model.AllView>
          <owner class="hudson" reference="../../.."/>
          <name>all</name>
          <filterExecutors>false</filterExecutors>
          <filterQueue>false</filterQueue>
          <properties class="hudson.model.View$PropertyList"/>
        </hudson.model.AllView>
      </views>
      <primaryView>all</primaryView>
      <slaveAgentPort>50000</slaveAgentPort>
      <label></label>
      <crumbIssuer class="hudson.security.csrf.DefaultCrumbIssuer">
        <excludeClientIPFromCrumb>false</excludeClientIPFromCrumb>
      </crumbIssuer>
      <nodeProperties/>
      <globalNodeProperties/>
    </hudson>

修改 系统管理 > 全局安全配置 > 代理 修改端口为50000

> 不改端口也会导致slave agent 起不来

![format_png 11][]

调整完毕 执行commit

docker commit -a "build success" be59cef93de7    registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master0407
     docker push registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:master0407

然后修改test-jenkins-deploy.yaml的镜像为最新镜像

# 应用新的master镜像
    kubectl apply -f test-jenkins-deploy.yaml --force

### 4 创建jenkins-slave镜像 ###

> 因为slave官方镜像commit之后 无法和master连接 就自己做了镜像
> 
> slave镜像不支持中文 可以通过env添加

准备的东西如下

1 Dockerfile

2 java解压好的文件夹(上面的jdk安装包或者别的机子上复制过来都可以)

3 jenkins-slave

4 libltdl.so.7

5 slave.jar

6 .ssh 文件 用于下载ssh 的git代码 以及config文件 用于避免下载代码的时候交互yea/no 否则可能会卡在git clone

![format_png 12][]

Dockerfile 如下

FROM openshift/base-centos7
    # FROM centos:centos7.4.1708 换成这个应该可以支持中文
    MAINTAINER PDABC Enterprise Container Images <jiaminxu@hz-health.cn>
    # COPY maven /usr/share/maven
    COPY java       /usr/local/java
    COPY libltdl.so.7 /usr/lib64/libltdl.so.7
    COPY slave.jar /usr/share/jenkins/slave.jar
    COPY jenkins-slave /usr/local/bin/jenkins-slave
    COPY .ssh /root/.ssh
    
    # ENV LANG=en_US.UTF-8
    # ENV LANGUAGE=en_US.UTF-8
    ENV HOME /root/.jenkins
    ENV AGENT_WORKDIR=/root/.jenkins/agent
    ENV JAVA_HOME /usr/local/java
    ENV MAVEN_HOME /usr/local/maven/
    ENV CLASSPATH .:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
    ENV PATH ${JAVA_HOME}/bin:${MAVEN_HOME}/bin:${PATH}
    ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"
    
    RUN  chmod 755 /usr/share/jenkins \
      && chmod 644 /usr/share/jenkins/slave.jar
    
    RUN mkdir -p /root/.jenkins \
     && mkdir -p ${AGENT_WORKDIR} \
     && yum -y install git subversion sshpassi maven
    
    VOLUME /root/.jenkins
    VOLUME ${AGENT_WORKDIR}
    WORKDIR /root/.jenkins
    
    USER root
    ENTRYPOINT ["jenkins-slave"]

build 上传

docker build -t registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:jnlp6.09 .
    docker push registry.cn-shanghai.aliyuncs.com/yjk-datag/jenkins:jnlp6.09

### 5 创建一个job或者scp一个job 到jenkins中 ###

# 我这里比较偷懒 直接scp了一个过来
    scp -r 192.168.1.171:/root/.jenkins/jobs/test-yjk-k8s /root/.jenkins/jobs/

然后reload jenkins

http://\{node\_ip\}:Nodeport/reload

![format_png 13][]

reload之后 scp过来的job会加载出来

job的pipeline配置如下

> 这里需要修改node为node(‘jenkins-jnlp’) ，带上标签，而且我注释了部署脚本，只执行了agent编译。后期需要部署 还需要nfs挂载部署脚本的文件夹。

// Using git without checkout 
    pipeline {
      agent any
      parameters {
        gitParameter branchFilter: 'origin/(.*)', defaultValue: 'master', name: 'Branch', sortMode: 'DESCENDING_SMART', type: 'PT_BRANCH'
      }
    
      stages {
            // stage("deleteDir") {
            //     steps{
            //         script{
            //             println env.WORKSPACE
            //             //dir("${env.WORKSPACE}/testdata"){   //切换到当前工作目录下的testdata目录
            //             //   sh "pwd"                         //sh命令可以 sh "command..." 也可以 sh("command....")
            //             }
            //             sh("ls -al ${env.WORKSPACE}")
            //             deleteDir()  // clean up current work directory   //清空目录
            //             sh("ls -al ${env.WORKSPACE}")
            //         }
            //     }
        stage('更新拉取最新代码') {
          steps {
            git branch: "${params.Branch}", url: 'git@gitlab.ihaozhuo.com:Java_Service/YJK-Java.git'
          }
        }
        stage('sonar扫描') {
          steps {
            sh "echo ${WORKSPACE} "
            sh "echo ${JOB_NAME} "
            //sh "echo ${deploy_step} "
            sh "echo ${env.JOB_NAME} "
            script
            {
                switch(SONAR_SCAN)
                {
                    case "TRUE":
                        sh "echo 扫描"
                        sh "sh  /srv/deploy_scripts/do_sonar.sh ${WORKSPACE} ${JOB_NAME}"
                        // sh "echo \u001B[31m不扫描\u001B[0m ！！！"
                    default:
                        sh "echo \u001B[31m不扫描\u001B[0m ！！！"
                        // sh "sh  /srv/deploy_scripts/do_sonar.sh ${WORKSPACE} ${JOB_NAME}"
                
            }
            }
                }
    
          }
        stage('输出项目的参数') {
          steps {
            sh "echo ${WORKSPACE} "
            sh "echo ${Branch} "
            //sh "echo ${deploy_step} "
            sh "echo ${env.JOB_NAME} "
            script {
                wrap([$class: 'BuildUser']) {
                //发布war包到指定服务器，虚拟机文件目录通过shell脚本初始化建立，所以目录是固定的
                //sh "sshpass -p ${serverPasswd} scp ${params.warLocation} ${serverName}@${serverIP}:htdocs/war"
                //这里增加了一个小功能，在服务器上记录了基本部署信息，方便多人使用一套环境时问题排查，storge in {WORKSPACE}/deploy.log  & remoteServer:htdocs/war
                //Date date = new Date()
                def deploylog="${BUILD_USER} use pipeline  '${JOB_NAME}(${BUILD_NUMBER})' "
                println deploylog
                //sh "echo ${deploylog} >>${WORKSPACE}/deploy.log"
                //sh "sshpass -p ${serverPasswd} scp ${WORKSPACE}/deploy.log ${serverName}@${serverIP}:htdocs/war"
                //jetty restart，重启jetty
                //sh "sshpass -p ${serverPasswd} ssh ${serverName}@${serverIP} 'bin/jettyrestart.sh' "
                buildName "#${BUILD_NUMBER}-^${BRANCH}^-${BUILD_USER}"
                HTTPD_LOCATION= sh(  returnStdout: true, script: 'git show -s  |grep -vE "commit|Date" |grep -v "^$" |sed "s/^[ \\t]*//g"')
                //echo "HTTPD_LOCATION =${HTTPD_LOCATION}"
                //def color_model="<font color=\"red\">$Model </font>"
                
                buildDescription  "${HTTPD_LOCATION} ${BUILD_USER}此次部署项目: ${Model} " 
                }
            }
            script
            {
                switch(Model)
                {
                    case "all":
                        sh "echo \u001B[31m部署全部模块\u001B[0m ！！！"
                    default:
                        sh "echo others"
                
            }
            }
                }
    
          }
        }
    
      }
    
    // 这个暂时没用   
    def doOneProject(p_name){
       stage('当前执行工程') {
         echo "${p_name}"
       }
       stage('工程执行结果') {
          echo "结束了${p_name}"
       }
    }
    
    node('jenkins-jnlp') {
        timestamps {
        env.PRO_TYPE = "test"
        ansiColor('xterm') {
        stage('Java编译') { 
            sh "echo ${WORKSPACE} "
            sh "echo ${Branch} "
            //sh "echo ${deploy_step} "
            sh "echo ${env.JOB_NAME} "
            script
            {
                switch(Model)
                {
                    case "yjk-all":
                        sh "echo \u001B[31m部署全部模块\u001B[0m ！！！"
                        sh "cd ${WORKSPACE} && rm dev-properties/ -rf && git clone git@gitlab.ihaozhuo.com:dev-properties/dev-properties.git  &&  cd ${WORKSPACE}/haozhuo/ && mvn clean install -T 1C -Dmaven.test.skip=true  -Dmaven.compile.fork=true -Ptest -Dautoconfig.userProperties=${WORKSPACE}/dev-properties/test1.properties -Dautoconfig.charset=utf-8 -Dautoconfig.interactive=off"
                    default:
                        sh "echo 部署  \u001B[32m $Model \u001B[0m !!!"
                        sh "cd ${WORKSPACE} && rm dev-properties/ -rf && git clone git@gitlab.ihaozhuo.com:dev-properties/dev-properties.git  &&  cd ${WORKSPACE}/haozhuo/ && mvn clean install  -pl $model -am -U  -T 1C -Dmaven.test.skip=true  -Dmaven.compile.fork=true -Ptest -Dautoconfig.userProperties=${WORKSPACE}/dev-properties/test1.properties -Dautoconfig.charset=utf-8 -Dautoconfig.interactive=off"
                
            }
            }
       }
       stage('执行数据库脚本') { 
            // sh "/bin/bash /srv/yaml/autosql.sh  redis  $version"
            // sh "/bin/bash /srv/yaml/autosql.sh  mysql  $version"
            // sh "/bin/bash /srv/yaml/autosql.sh  mongodb  $version"
    
    
       }
        stage('统一构建镜像并发布') { 
            sh "echo ${Model}"
            sh "echo ${WORKSPACE} "
            sh "echo ${env.PRO_TYPE} "
            //sh "echo ${deploy_step} "
            sh "echo ${BUILD_NUMBER} "
            sh "echo sh /srv/deploy_scripts/k8s_deploy.sh $Model $WORKSPACE ${env.PRO_TYPE}  $BUILD_NUMBER ${env.PRO_TYPE}"
            sh "echo build successful"
    
       }
    
    
    }
    }
    
    }

### 6 构建job测试及结果 ###

因为挂载了m2仓库 所以下载依赖会比较少 部署比较快

![format_png 14][]

![format_png 15][]

![format_png 16][]

![format_png 17][]

![format_png 18][]

至此部署成功 如果有需要 可以再commit一次。

[format_png]: /images/20221123/0cf01a3c5c2b4f4bb44f41dd3e13aa84.png
[format_png 1]: /images/20221123/c65baa8d54b5438b953898e5837917d4.png
[format_png 2]: /images/20221123/0fd2b1eed98c4fb3a47cecdb67977cee.png
[format_png 3]: /images/20221123/d1f0972737c240768a41d7ebdef78a21.png
[format_png 4]: /images/20221123/86aecca0256942ea8ce27ae97a9c8a6e.png
[format_png 5]: /images/20221123/f5b9f5a2f4f648c0b6b9397eedef2b78.png
[format_png 6]: /images/20221123/26a0310982534210bb7876810d7d184d.png
[format_png 7]: /images/20221123/f90469fec8634fca8c526c5a89877700.png
[format_png 8]: /images/20221123/fe2c70def9af4d30b87bc157745b9c1f.png
[format_png 9]: /images/20221123/587e85f7eaa2455097232a0c9cbd18cc.png
[format_png 10]: /images/20221123/90bae95159014d268aeee80ddac86523.png
[format_png 11]: /images/20221123/672adac30e894b2794493c84edd8eb30.png
[format_png 12]: /images/20221123/9cb5d8a5e5be44e19cbbbd1fb0a80dce.png
[format_png 13]: /images/20221123/8c198c4c182e48969c1487169c7debae.png
[format_png 14]: /images/20221123/4d9b5ebf158e41e0865d6128801ead14.png
[format_png 15]: /images/20221123/505db90f3e0441648d572c2fc7bf25df.png
[format_png 16]: /images/20221123/15439bbe87e14ce492d302ba4414a60a.png
[format_png 17]: /images/20221123/0ec01c1b423547fc885f67d84d81c798.png
[format_png 18]: /images/20221123/78633de6c4ca4bda820f263762476f8e.png