# 使用k8s部署项目的流程 ## 容器交付流程 ![image-20220205084826604](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-f4eac9d755819af1642f6148b25f7362e6f8bfd2%2Fimage-20220205084826604.png?alt=media) k8s部署项目的流程： ![image-20220205085018910](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-e6d5eca5d02d916ca5ad56797be8a4c67ab1d78e%2Fimage-20220205085018910.png?alt=media) ## 案例：交付一个java应用程序 项目地址： ### 1. 制作镜像 #### 1.1 编写dockerfile 使用dockerfile的多阶段构建 ```dockerfile FROM alpine/git:v2.32.0 as cloner MAINTAINER "yangsx.yangsx95@qq.com" WORKDIR /opt # 拉取代码 RUN git clone https://github.com/lizhenliang/tomcat-java-demo.git FROM maven:3.8-openjdk-8 as builder MAINTAINER "yangsx.yangsx95@qq.com" WORKDIR /opt # 将cloner阶段的opt/tomcat-java-demo这个文件拷贝到当前阶段的/opt/路径下 COPY --from=cloner /opt/tomcat-java-demo/ /opt/ # 添加阿里云maven镜像 RUN echo '/usr/share/maven/ref/repositoryaliyunmaven*阿里云公共仓库https://maven.aliyun.com/repository/public' > /opt/settings.xml # 执行构建 RUN mvn clean package --settings=/opt/settings.xml -Dmaven.test.skip=true FROM tomcat:8-jdk8 as runner MAINTAINER "yangsx.yangsx95@qq.com" WORKDIR /usr/local/tomcat/webapps/ COPY --from=builder /opt/target/*.war /usr/local/tomcat/webapps/ROOT.war ``` #### 1.2 构建镜像到本地 ``` docker build -t yangsx95/hello-girl . ``` 测试镜像： ``` docker run --rm -it yangsx95/hello-girl -p 8080:8080 ``` #### 1.3 创建harbor项目 ![image-20220206160543078](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-6992c9e1d10709e1c908fd4c9e74378b18025547%2Fimage-20220206160543078.png?alt=media) ![·](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-6f536fb865bcd8045436fba2979c033cff1004c4%2Fimage-20220206160813640.png?alt=media) ![image-20220206160828222](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-6186e3d0c83caa28acdd4d036bd22619e6eb16f6%2Fimage-20220206160828222.png?alt=media) #### 1.4 登录harbor ``` docker login 192.168.121.67 ``` > 注意，如果使用HTTP访问harbor，需要将harbor加入到docker的http仓库白名单中 > > ```shell > vim /etc/docker/daemon.json > { > "insecure-registries": ["ip:端口"] > } > ``` #### 1.5 推送镜像到harbor 1. 在habor界面可以看到镜像推送的命令 ![image-20220206162035724](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-c63df2d9a5544d739f9f2c9907e945942c55cfe6%2Fimage-20220206162035724.png?alt=media) 2. 给镜像打tag，并推送到harbor ``` docker tag yangsx95/hello-girl:latest 192.168.121.67/hello-girl/hello-girl:v1 docker push 192.168.121.67/hello-girl/hello-girl:v1 ``` 3. 查看harbor项目 ![image-20220206162249579](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-7dafc3598feb2dca98c4fb94d9e6e5b3a88ac0e3%2Fimage-20220206162249579.png?alt=media) #### 1.6 配置harbor凭据到k8s Secret中 将harbor的登录凭据保存到k8s Secret中，以供k8s服务拉取镜像使用： ```shell kubectl create secret docker-registry harbor-admin \ --docker-username=harbor \ --docker-password=Harbor12345 \ --docker-email=yangsx95@qq.com \ --docker-server=192.168.121.67 ``` 可以通过如下方式在拉取镜像时使用： ```yaml imagePullSecrets: - name: registry-auth ``` 使用`kubectl get secrets`可以查看Secret是否被正常创建： ``` kubectl get secrets NAME TYPE DATA AGE default-token-zl5r6 kubernetes.io/service-account-token 3 5d1h harbor-admin kubernetes.io/dockerconfigjson 1 6s nfs-client-provisioner-token-sntk5 kubernetes.io/service-account-token 3 2d14h ``` ### 2. 使用控制器部署镜像 #### 2.1 部署项目依赖的Mysql数据库 注意，不建议在k8s中部署数据库： 1. Mysql 属于重量级应用（负载高、吃资源、部署复杂） 2. Mysql部署难度大，主要针对集群方式，每个集群的角色不同，需要使用StatefulSet 3. Mysql是比较稳定的服务，极少会更新，享受不到k8s的特性 4. 对服务性能会有一定的衰减，但是Mysql对性能要求很高 5. 可以在k8s中部署单节点，可以享受k8s的快速部署 *** **创建如下mysql deployment资源清单文件 `mysql.yml`（mysql5.7版本，不支持arm64）：** ```yaml # 创建Secret，为准备数据库密码配置 apiVersion: v1 kind: Secret metadata: name: java-demo-db namespace: default type: Opaque data: mysql-root-password: "MTIzNDU2" mysql-password: "MTIzNDU2" --- # 创建Deployment部署Mysql apiVersion: apps/v1 kind: Deployment metadata: name: java-demo-db namespace: default spec: selector: matchLabels: project: www app: mysql template: metadata: labels: project: www app: mysql spec: containers: - name: db image: mysql:5.7.30 resources: requests: cpu: 500m memory: 512Mi limits: cpu: 500m memory: 512Mi env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: java-demo-db key: mysql-root-password - name: MYSQL_PASSWORD valueFrom: secretKeyRef: name: java-demo-db key: mysql-password - name: MYSQL_USER value: "yangsx" - name: MYSQL_DATABASE value: "k8s" ports: - name: mysql containerPort: 3306 livenessProbe: exec: command: - sh - -c - "mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}" initialDelaySeconds: 30 periodSeconds: 10 readinessProbe: exec: command: - sh - -c - "mysqladmin ping -u root -p${MYSQL_ROOT_PASSWORD}" initialDelaySeconds: 5 periodSeconds: 10 volumeMounts: - name: data mountPath: /var/lib/mysql volumes: - name: data persistentVolumeClaim: claimName: java-demo-db --- # 配置PVC，使用nfs动态分配PV apiVersion: v1 kind: PersistentVolumeClaim metadata: name: java-demo-db namespace: default spec: storageClassName: "managed-nfs-storage" accessModes: - "ReadWriteOnce" resources: requests: storage: "8Gi" --- # 配置Service为集群其他pod提供服务 apiVersion: v1 kind: Service metadata: name: java-demo-db namespace: default spec: type: ClusterIP ports: - name: mysql port: 3306 targetPort: mysql selector: project: www app: mysql ``` *** **执行部署并查看部署结果：** ```shell $ kubectl apply -f mysql.yaml $ kubectl get svc java-demo-db NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE java-demo-db ClusterIP 10.103.36.119 3306/TCP 2m19s ``` **测试mysql是否正常连接（使用mysql-client镜像）：** ```shell $ kubectl run mysql-client --rm -it --image=mysql:5.7.30 -- bash If you don't see a command prompt, try pressing enter. root@mysql-client:/# mysql -h 10.101.140.88 -uroot -p123456 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 99 Server version: 5.7.30 MySQL Community Server (GPL) Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | k8s | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.07 sec) ``` #### 2.2 导入项目sql文件 准备sql文件： 将该sql文件下载并拷贝到mysql-client中： ```shell kubectl cp tables_ly_tomcat.sql mysql-client:/ ``` 在mysql-client中使用mysql命令导入到数据库中： ``` mysql> create database test; Query OK, 1 row affected (0.09 sec) mysql> use test; Database changed mysql> source /tables_ly_tomcat.sql Query OK, 1 row affected, 1 warning (0.02 sec) Database changed Query OK, 0 rows affected (0.12 sec) Query OK, 0 rows affected (0.00 sec) mysql> show tables; +----------------+ | Tables_in_test | +----------------+ | user | +----------------+ 1 row in set (0.01 sec) ``` #### 2.3 创建ConfigMap保存项目配置文件 创建一个ConfigMap，它拥有一个 `application.yml`键，用于对应spring-boot项目的根配置，其内容拷贝自 ： ```yaml apiVersion: v1 kind: ConfigMap metadata: name: java-demo-config data: application.yml: | server: port: 8080 spring: datasource: url: jdbc:mysql://localhost:3306/test?characterEncoding=utf-8 username: root password: 123456789 driver-class-name: com.mysql.jdbc.Driver freemarker: allow-request-override: false cache: true check-template-location: true charset: UTF-8 content-type: text/html; charset=utf-8 expose-request-attributes: false expose-session-attributes: false expose-spring-macro-helpers: false suffix: .ftl template-loader-path: - classpath:/templates/ ``` #### 2.4 创建deployment ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-demo spec: replicas: 1 selector: matchLabels: project: www app: java-demo template: metadata: labels: project: www app: java-demo spec: # 指定镜像拉取的所需要的secret imagePullSecrets: - name: registry-auth containers: - image: 192.168.121.67/microservice/java-demo:v1 name: java-demo volumeMounts: - name: config mountPath: "/usr/local/tomcat/webapps/ROOT/WEB-INF/classes/application.yml" subPath: application.yml resources: requests: cpu: 0.5 memory: 500Mi limits: cpu: 1 memory: 1Gi livenessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 50 periodSeconds: 10 readinessProbe: httpGet: path: / port: 8080 initialDelaySeconds: 50 periodSeconds: 10 volumes: - name: config configMap: name: java-demo-config items: - key: "application.yml" path: "application.yml" ``` **一般必须配置的几个属性：** 1. 环境变量 2. 资源配额 （重要） 3. 健康检查 4. 卷挂载点 ### 3. 对外暴露应用 #### 3.1 配置Service ```yaml apiVersion: v1 kind: Service metadata: name: java-demo spec: selector: project: www app: java-demo ports: - protocol: TCP port: 80 targetPort: 8080 ``` #### 3.2 配置Ingress ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: java-demo annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: hello.girl.com # 将域名映射到 java-web 服务 http: paths: - path: / pathType: Prefix backend: service: name: java-web # 将所有请求发送到 java-web 服务的 8080 端口 port: number: 8080 ``` #### 3.3 增加公网负载均衡 ![image-20220208104847193](https://2351062869-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7b2CdwBN9liniVJpfEAc%2Fuploads%2Fgit-blob-3a42e1b84a9af735092189d8abf7a1de2add7138%2Fimage-20220208104847193.png?alt=media) ```nginx upstream ingress-controller { server 192.168.121.11:80; server 192.168.121.12:80; } server { listen 80; server_name: _; location /{ proxy_pass http://ingress-controller; proxy_set_header Host $Host; # $Host是浏览器访问网址时指定的域名 } } ```