手把手教你寫一個通用的helm chart

[TOC]

1. 模板介紹

首先，放上此模板鏈接：

https://github.com/ygqygq2/charts/tree/master/ygqygq2/mod-chart

此chart可當作POD單image的通用模板，只需要使用sed替換下chart名，并修改下README.md和NOTES.txt就可以了。下文，我通過復制此chart成example-chart來作示范說明。

[root@master1 mod-chart]# tree
.
├── Chart.yaml  # chart版本信息文件
├── README.md  # chart說明文件
├── templates  # kubernetes資源yaml模板
│?? ├── configmap.yaml  # configmap模板
│?? ├── deployment-statefulset.yaml  # deployment或statefulset模板
│?? ├── _helpers.tpl  # 輔助模板和 partials
│?? ├── ingress.yaml  # ingress模板
│?? ├── NOTES.txt  # 部署chart后輸出的幫助文檔
│?? ├── pvc.yaml  # pvc模板
│?? ├── secret.yaml  # secret模板
│?? ├── service-headless.yaml  # service headless模板
│?? └── service.yaml  # service模板
└── values.yaml  # 默認設置

1 directory, 12 files
[root@master1 mod-chart]# helm3 lint --strict .
1 chart(s) linted, 0 chart(s) failed

2. 新chart制作

注：
下文中文件內容我保留，只加注釋。
注釋中需要修改的地方 [*] 標記為必選，[-] 標識為可選。

2.1 目錄準備

將模板mod-chart復制成example-chart，并作內容替換。

rsync -avz mod-chart/ example-chart/
cd example-chart/
sed -i 's@mod-chart@example-chart@g' *.*
sed -i 's@mod-chart@example-chart@g' templates/*.*

2.2 修改Chart.yaml

vim Chart.yaml

apiVersion: v1  # 當前helm api版本，不需要修改
appVersion: 1.14.2  # 此處為你應用程序的版本號 [*]
description: Chart for the nginx server  # 介紹此chart是干嘛的，按需求修改
engine: gotpl  # go模板引擎，不需要修改 [-]
name: example-chart  # 模板名，對應目錄名 [*]
version: 1.0.0  # 此chart版本號 [*]
home: http://www.nginx.org  # 應用程序官網 [*]
icon: https://cache.yisu.com/upload/information/20200309/33/60313.jpg  # 應用程序logo地址 [*]
keywords:  # 關鍵字列表 [*]
- nginx
- http
- web
- www
- reverse proxy
maintainers:  # 維護人員列表 [*]
- email: 29ygq@sina.com
  name: Chinge Yang
sources:  # 應用程序來源 [-]
- https://github.com/bitnami/bitnami-docker-nginx

2.3 修改values.yaml

因為values.yaml設置涉及到yaml格式，yaml文件格式說明可以看這篇文章：

http://www.ruanyifeng.com/blog/2016/07/yaml.html

這里提幾個常用的地方：

1. 使用2個空格作縮進；
2. 確認數字為字符類型時，使用雙引號引起來；
3. 為了迎合helm3的規范，空定義最好將相關符號補上：

   string: ""
   list: []
   map: {}

沒什么特殊要求，一般需要修改的地方有image、service、healthCheck、persistentVolume.mountPaths

# Default values for mod-chart.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

## Global Docker image parameters
## Please, note that this will override the image parameters, including dependencies, configured to use the global value
## Current available global Docker image parameters: imageRegistry and imagePullSecrets
##
global:  # 設置后覆蓋后面默認的鏡像倉庫
  imageRegistry: ""
  imagePullSecrets: []
#     - myRegistryKeySecretName

statefulset:
  enabled: false

## String to partially override fullname template (will maintain the release name)
##
nameOverride: ""

## String to fully override fullname template
##
fullnameOverride: ""

## By default deploymentStrategy is set to rollingUpdate with maxSurge of 25% and maxUnavailable of 25% .
## You can change type to `Recreate` or can uncomment `rollingUpdate` specification and adjust them to your usage.
deploymentStrategy: {}
  # rollingUpdate:
  #   maxSurge: 25%
  #   maxUnavailable: 25%
  # type: RollingUpdate

# 副本個數
replicaCount: 1

# 容器image及tag
image:
  registry: docker.io
  repository: bitnami/nginx
  tag: latest
  pullPolicy: IfNotPresent  # IfNotPresent: 有則不拉（減少流量和操作步驟），Always: 不管tag總拉（適合tag不變時更新）
  pullSecrets: []
  #  - private-registry-key

service:
  type: ClusterIP  # 一般不用修改
  ingressPort: 8080
  ports:
    http:  # 多端口暴露時，復制一段
      port: 8080  # Service port number for client-a port.
      protocol: TCP  # Service port protocol for client-a port.

## env set
## ref: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/
env: []
#  - name: DEMO_GREETING
#    value: "Hello from the environment"
#  - name: DEMO_FAREWELL
#    value: "Such a sweet sorrow"

## command set
startCommand: []
#  - "java -Xdebug -Xnoagent -Djava.compiler=NONE"
#  - "-Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n"
#  - "-Djava.security.egd=file:/dev/urandom"
#  - "-jar /test.jar"
#  - "-Duser.timezone=GMT+08"

## Enable configmap and add data in configmap
config:
  enabled: false
  subPath: ""
  mountPath: /conf
  data: {}

############################# 示例 ####################################
## 以下示例，掛載文件至 /conf/app.conf
#  enabled: true
#  mountPath: /conf/app.conf  
#  subPath: app.conf    # 使用subPath時，上面mountPath路徑寫文件完整絕對路徑
#  data:
#    app.conf: |-
#      appname = example-chart

## 以下示例，掛載多個文件至 /conf/ 下
#  enabled: true
#  mountPath: /conf    # 不使用subPath
#  data:
#    app.conf: |-
#      appname = example-chart
#    bpp.conf: |-
#      bppname
#
## 掛載多個文件至多個不同路徑，需要相應修改 templates/deployment-statefulset.yaml 
############################# 示例 ####################################

## To use an additional secret, set enable to true and add data
## 用法同上，不另作說明
secret:
  enabled: false
  mountPath: /etc/secret-volume
  subPath: ""
  readOnly: true
  data: {} 

## liveness and readiness 
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
healthCheck:
  enabled: true
  type: tcp  # http/tcp
  port: http  # 健康檢查的端口名或端口
  httpPath: '/'  # http時必須設置
  livenessInitialDelaySeconds: 10  # 初始延遲秒數
  livenessPeriodSeconds: 10  # 檢測周期，默認值10，最小為1
  readinessInitialDelaySeconds: 10  # 初始延遲秒數
  readinessPeriodSeconds: 10   # 檢測周期，默認值10，最小為1

resources: {}
  # 容器資源設置
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

## Node labels and tolerations for pod assignment
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
### ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
labels: {}
podAnnotations: {}
nodeSelector: {}
tolerations: []
affinity: {}
annotations: {}

## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistentVolume:   # 是否存儲持久化
  enabled: false
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, azure-disk on
  ##   Azure, standard on GKE, AWS & OpenStack)
  ##
  storageClass: "-"
  accessMode: ReadWriteOnce
  annotations: {}
  #   helm.sh/resource-policy: keep
  size: 1Gi  # 大小
  existingClaim: {}  # 使用已存在的pvc
  mountPaths: []
  #  - name: data-storage
  #    mountPath: /config
  #    subPath: config  # 多個路徑使用同一個pvc使用subPath，用法同上面config中示例說明
  #  - name: data-storage
  #    mountPath: /data
  #    subPath: data

ingress:  # 是否使用nginx暴露域名或端口
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: []

## Prometheus Exporter / Metrics
##
metrics:
  enabled: false
  image:
    registry: docker.io
    repository: nginx/nginx-prometheus-exporter
    tag: 0.1.0
    pullPolicy: IfNotPresent
    ## Optionally specify an array of imagePullSecrets.
    ## Secrets must be manually created in the namespace.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    ##
    pullSecrets: []
    #   - myRegistrKeySecretName
  ## Metrics exporter pod Annotation and Labels
  podAnnotations:
    # prometheus.io/scrape: "true"
    # prometheus.io/port: "9113"
    ## Metrics exporter resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    ##
  resources: {}

## Uncomment and modify this to run a command after starting the core container.
## ref: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
lifecycle: {}
  # preStop:
  #   exec:
  #     command: ["/bin/bash","/pre-stop.sh"]
  # postStart:
  #   exec:
  #     command: ["/bin/bash","/post-start.sh"]

## Deployment additional volumes.
deployment:
  additionalVolumes: []

## init containers
## ref: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
## Add init containers. e.g. to be used to give specific permissions for data
## Add your own init container or uncomment and modify the given example.
initContainers: {}
#  - name: fmp-volume-permission
#    image: busybox
#    imagePullPolicy: IfNotPresent
#    command: ['chown','-R', '200', '/extra-data']
#    volumeMounts:
#      - name: extra-data
#        mountPath: /extra-data

## Additional containers to be added to the core pod.
additionalContainers: {}
#  - name: my-sidecar
#    image: nginx:latest
#  - name: lemonldap-ng-controller
#    image: lemonldapng/lemonldap-ng-controller:0.2.0
#    args:
#      - /lemonldap-ng-controller
#      - --alsologtostderr
#      - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration
#    env:
#      - name: POD_NAME
#        valueFrom:
#          fieldRef:
#            fieldPath: metadata.name
#      - name: POD_NAMESPACE
#        valueFrom:
#          fieldRef:
#            fieldPath: metadata.namespace
#    volumeMounts:
#    - name: copy-portal-skins
#      mountPath: /srv/var/lib/lemonldap-ng/portal/skins

2.4 修改README.md和templates/NOTES.txt

根據 values.yaml中的默認設置相應修改README.md，內容使用markdown語法，這里不作詳細說明。
templates/NOTES.txt是部署chart后輸出的幫助文檔，里面支持go template語法。模板里已經寫成了非常通用。必要情況下，適當按應用需求來修改，這樣會顯得部署后提示非常友好和人性化。

2.5 templates下yaml簡要說明

templates目錄下為kubernetes資源yaml文件模板，以資源名命名文件名，復雜些的可以加上資源功能或者模塊名等。

templates/secret.yaml

{{- if .Values.secret.enabled }}  # if用法。語法代碼注意形成一致縮進習慣，便于閱讀
apiVersion: v1
kind: Secret
metadata:
  name: {{ template "example-chart.fullname" . }}  # 使用輔助模板時，點號可用 $ 代替，特別是在range下
  labels:
    app: {{ template "example-chart.name" . }}
    chart: {{ template "example-chart.chart" . }}
    release: "{{ .Release.Name }}"
    heritage: "{{ .Release.Service }}"
    {{- if .Values.labels }}
{{ toYaml .Values.labels | indent 4 }}  # 使用toYaml時，不縮進，indent接空格數
    {{- end }}
data:
{{- range $key, $value := .Values.secret.data }}  # range用法
  {{ $key }}: {{ $value | b64enc | quote }}  # secret中value注意使用base64轉換
{{- end }}
{{- end }}

templates/deployment-statefulset.yaml

{{- if .Values.statefulset.enabled }}  # 判斷使用deployment和statefulset資源api類型
apiVersion: apps/v1
kind: StatefulSet
{{- else }}
apiVersion: apps/v1
kind: Deployment
{{- end }}
metadata:
  name: {{ template "example-chart.fullname" . }}
  labels:
    app: {{ template "example-chart.name" . }}
    chart: {{ template "example-chart.chart" . }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
{{- if .Values.labels }}  # 額外的標簽
{{ toYaml .Values.labels | indent 4 }}
{{- end }}
{{- if .Values.annotations }}  # 自定義注釋
  annotations:
{{ toYaml .Values.annotations | indent 4 }}
{{- end }}
spec:
  replicas: {{ .Values.replicaCount }}  # 副本數
  {{- if .Values.statefulset.enabled }}  # statefulset需要定義serviceName
  serviceName: {{ template "example-chart.fullname" . }}-headless
  {{- end }}
  {{- if .Values.deploymentStrategy }}
  strategy:
{{ toYaml .Values.deploymentStrategy | indent 4 }}
  {{- end }}
  selector:
    matchLabels:
      app: {{ template "example-chart.name" . }}
      release: {{ .Release.Name }}
  template:
    metadata:
      annotations:
  {{- if .Values.podAnnotations }}
{{ toYaml .Values.podAnnotations | indent 8 }}
{{- end }}
{{- if .Values.metrics.podAnnotations }}
{{ toYaml .Values.metrics.podAnnotations | indent 8 }}
{{- end }}
      labels:
        app: {{ template "example-chart.name" . }}
        release: {{ .Release.Name }}
    spec:
{{- include "example-chart.imagePullSecrets" . | indent 6 }}
      {{- if .Values.initContainers }}
      initContainers:
{{ toYaml .Values.initContainers | indent 8 }}
      {{- end }}
      nodeSelector:
{{ toYaml .Values.nodeSelector | indent 8 }}
      affinity:
{{ toYaml .Values.affinity | indent 8 }}
      tolerations:
{{ toYaml .Values.tolerations | indent 8 }}
      containers:
{{- if .Values.metrics.enabled }}  # metrics容器可根據需求修改
        - name: metrics
          image: {{ template "metrics.image" . }}
          imagePullPolicy: {{ .Values.metrics.image.pullPolicy | quote }}
          command: [ '/usr/bin/exporter', '-nginx.scrape-uri', 'http://127.0.0.1:8080/status']
          ports:
          - name: metrics
            containerPort: 9113
          livenessProbe:
            httpGet:
              path: /metrics
              port: metrics
            initialDelaySeconds: 15
            timeoutSeconds: 5
          readinessProbe:
            httpGet:
              path: /metrics
              port: metrics
            initialDelaySeconds: 5
            timeoutSeconds: 1
          resources:
{{ toYaml .Values.metrics.resources | indent 12 }}
{{- end }}
        - name: {{ .Chart.Name }}
          image: {{ template "example-chart.image" . }}
          imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
          {{- if .Values.lifecycle }}
          lifecycle:
{{ toYaml .Values.lifecycle | indent 12 }}
          {{- end }}
          {{- if .Values.startCommand }}
          command:
{{ toYaml .Values.startCommand |indent 12 }}
          {{- end }}
          env:
{{ toYaml .Values.env | indent 12 }}
          resources:
{{ toYaml .Values.resources | indent 12 }}
          ports:
            {{- range $key, $value := .Values.service.ports }}
            - name: {{ $key }}
              containerPort: {{ $value.port }}
              protocol: {{ $value.protocol }}
            {{- end }}
          {{- if .Values.healthCheck.enabled }}
          livenessProbe:
            {{- if eq .Values.healthCheck.type "http" }}
            httpGet:
              path: {{ .Values.healthCheck.httpPath }}
              port: {{ .Values.healthCheck.port }}
            {{- else }}
            tcpSocket:
              port: {{ .Values.healthCheck.port }}
            {{- end }}
            initialDelaySeconds: {{ .Values.healthCheck.livenessInitialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.livenessPeriodSeconds }}
          readinessProbe:
            {{- if eq .Values.healthCheck.type "http" }}
            httpGet:
              path: {{ .Values.healthCheck.httpPath }}
              port: {{ .Values.healthCheck.port }}
            {{- else }}
            tcpSocket:
              port: {{ .Values.healthCheck.port }}
            {{- end }}
            initialDelaySeconds: {{ .Values.healthCheck.readinessInitialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.readinessPeriodSeconds }}
          {{- end }}
          volumeMounts:  # 容器掛載點
            {{- if .Values.config.enabled }}
            - name: {{ template "example-chart.name" . }}-conf
              mountPath: {{ .Values.config.mountPath }}
              subPath: {{ .Values.config.subPath }}
            {{- end }}
            {{- if .Values.secret.enabled }}
            - name: {{ template "example-chart.name" . }}-secret
              mountPath: {{ .Values.secret.mountPath }}
              subPath: {{ .Values.secret.subPath }}
              readOnly: {{ .Values.secret.readOnly }}
            {{- end }}
{{- if .Values.persistentVolume.mountPaths }}
{{ toYaml .Values.persistentVolume.mountPaths | indent 12 }}
{{- end }}
        {{- if .Values.additionalContainers }}
{{ toYaml .Values.additionalContainers | indent 8 }}
        {{- end }}
      volumes:  # volume名需要和上文volumeMounts中的名字一一對應
        {{- if .Values.config.enabled }}
        - name: {{ template "example-chart.name" . }}-conf
          configMap:
            name: {{ template "example-chart.fullname" . }}
        {{- end }}
        {{- if .Values.secret.enabled }}
        - name: {{ template "example-chart.name" . }}-secret
          secret:
            secretName: {{ template "example-chart.fullname" . }}
        {{- end }}
        {{- if .Values.deployment.additionalVolumes }}
{{ toYaml .Values.deployment.additionalVolumes | indent 8 }}
        {{- end }}
{{- if not .Values.statefulset.enabled }}
          {{- if .Values.persistentVolume.enabled }}
        - name: data-storage
          persistentVolumeClaim:
            claimName: {{ .Values.persistentVolume.existingClaim | default (include "example-chart.fullname" .) }}
          {{- else }}
        - name: data-storage
          emptyDir: {}
          {{- end }}
{{- else }}
  {{- if .Values.persistentVolume.enabled }}
  volumeClaimTemplates:
  - metadata:
      name: data-storage
      labels:
        app: {{ template "example-chart.name" . }}
        chart: {{ template "example-chart.chart" . }}
        release: {{ .Release.Name }}
        heritage: {{ .Release.Service }}
    spec:
      accessModes:
        - {{ .Values.persistentVolume.accessMode | quote }}
      annotations:
      {{- range $key, $value := $.Values.persistentVolume.annotations }}
        {{ $key }}: {{ $value }}
      {{- end }}
      resources:
        requests:
          storage: {{ .Values.persistentVolume.size }}
          {{- if .Values.persistentVolume.storageClass }}
            {{- if (eq "-" .Values.persistentVolume.storageClass) }}
      storageClassName: ""
            {{- else }}
      storageClassName: "{{ .Values.persistentVolume.storageClass }}"
            {{- end }}
          {{- end }}
  {{- else }}
        - name: data-storage
          emptyDir: {}
  {{- end }}
{{- end -}}

3. 小結

以上yaml中未作詳細說明，仔細看其內容都能明白大致意思。以下是對于helm chart新手的一些建議：

1. 剛接觸helm chart時，多模仿stable和bianami中charts的寫法，特別是values.yaml和templates目錄中的一些設計，差不多都已經非常統一了。這樣遇到自己有相似需求，可直接使用相應的功能塊，寫出來的chart也顯得非常專業。
2. 使用chart時，最好使用helm命令fetch下來，大概讀一遍其chart內容，這樣看多了，自然就越來越熟悉，而且出錯時，也便于自己排查問題。

參考資料：
[1] https://helm.sh/
[2] https://whmzsu.github.io/helm-doc-zh-cn/