kubernetes集群安裝指南：kubelet組件部署

kubelet是kubernetes中一個重要的組件。對pod容器的管理 ，執行交互式命令(如 exec、run、logs 等)都離不開它，kubelet 運行在每個 worker 節點上，負責接收 kube-apiserver 發送的請求，kubelet 在啟動時會自動向 kube-apiserver 發送注冊信息，內置的 cadvisor 統計和監控節點的資源使用情況。

1. 準備工作

特別說明：這里所有的操作都是在devops這臺機器上通過ansible工具執行；kubelet在需要使用kubeconfig文件來認證訪問kube-apiserver，因此需要為其開啟證書輪轉

為確保安全，部署時關閉了 kubelet 的非安全 http 端口，對請求進行認證和授權，拒絕未授權的訪問(如 apiserver、heapster 的請求)。

環境變量定義

#################### Variable parameter setting ######################
KUBE_NAME=kubelet
K8S_INSTALL_PATH=/data/apps/k8s/kubernetes
K8S_BIN_PATH=${K8S_INSTALL_PATH}/sbin
K8S_LOG_DIR=${K8S_INSTALL_PATH}/logs
K8S_CONF_PATH=/etc/k8s/kubernetes
KUBE_CONFIG_PATH=/etc/k8s/kubeconfig
CA_DIR=/etc/k8s/ssl
SOFTWARE=/root/software
HOSTNAME=`hostname`
VERSION=v1.14.2
DOWNLOAD_URL=https://github.com/devops-apps/download/raw/master/kubernetes/kubernetes-server-${VERSION}-linux-amd64.tar.gz
ETH_INTERFACE=eth2
LISTEN_IP=$(ifconfig | grep -A 1 ${ETH_INTERFACE} |grep inet |awk '{print $2}')
USER=k8s
CLUSTER_DNS_DOMAIN=k8s.mo9.com
CLUSTER_DNS_IP=10.254.0.2
CLUSTER_PODS_CIDR=172.16.0.0/20

2 部署kubelet組件

2.1 安裝kubelet二進制文件

### 1.Check if the install directory exists.
if [ ! -d "$K8S_BIN_PATH" ]; then
     mkdir -p $K8S_BIN_PATH
fi

if [ ! -d "$K8S_LOG_DIR/$KUBE_NAME" ]; then
     mkdir -p $K8S_LOG_DIR/$KUBE_NAME
fi

if [ ! -d "$K8S_CONF_PATH" ]; then
     mkdir -p $K8S_CONF_PATH
fi

if [ ! -d "$KUBE_CONFIG_PATH" ]; then
     mkdir -p $KUBE_CONFIG_PATH
fi

### 2.Install kubelet binary of kubernetes.
if [ ! -f "$SOFTWARE/kubernetes-server-${VERSION}-linux-amd64.tar.gz" ]; then
     wget $DOWNLOAD_URL -P $SOFTWARE >>/tmp/install.log  2>&1
fi
cd $SOFTWARE && tar -xzf kubernetes-server-${VERSION}-linux-amd64.tar.gz -C ./
cp -fp kubernetes/server/bin/$KUBE_NAME $K8S_BIN_PATH
ln -sf  $K8S_BIN_PATH/${KUBE_NAME} /usr/local/bin
chmod -R 755 $K8S_INSTALL_PATH

2.3 創建kubelet配置文件

```# configure default system config
cat >${K8S_CONF_PATH}/kubelet-config.yaml <<EOF
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: "${LISTEN_IP}"
staticPodPath: ""
syncFrequency: 1m
fileCheckFrequency: 20s
httpCheckFrequency: 20s
staticPodURL: ""
port: 10250
readOnlyPort: 0
rotateCertificates: true
serverTLSBootstrap: true
authentication:
anonymous:
enabled: false
webhook:
enabled: true
cacheTTL: 2m0s
x509:
clientCAFile: "${CA_DIR}/ca.pem"
authorization:
mode: Webhook
registryPullQPS: 0
registryBurst: 20
eventRecordQPS: 0
eventBurst: 20
enableDebuggingHandlers: true
enableContentionProfiling: true
healthzPort: 10248
healthzBindAddress: "${LISTEN_IP}"
clusterDomain: "${CLUSTER_DNS_DOMAIN}"
clusterDNS:

• "${CLUSTER_DNS_IP}"
  nodeStatusUpdateFrequency: 10s
  nodeStatusReportFrequency: 1m
  imageMinimumGCAge: 2m
  imageGCHighThresholdPercent: 85
  imageGCLowThresholdPercent: 80
  volumeStatsAggPeriod: 1m
  kubeletCgroups: ""
  systemCgroups: ""
  cgroupRoot: ""
  cgroupsPerQOS: true
  cgroupDriver: cgroupfs
  runtimeRequestTimeout: 10m
  hairpinMode: promiscuous-bridge
  maxPods: 220
  podCIDR: "${CLUSTER_PODS_CIDR}"
  podPidsLimit: -1
  resolvConf: /etc/resolv.conf
  maxOpenFiles: 1000000
  kubeAPIQPS: 1000
  kubeAPIBurst: 2000
  serializeImagePulls: false
  evictionHard:
  memory.available: "100Mi"
  nodefs.available: "10%"
  nodefs.inodesFree: "5%"
  imagefs.available: "15%"
  evictionSoft: {}
  enableControllerAttachDetach: true
  failSwapOn: true
  containerLogMaxSize: 20Mi
  containerLogMaxFiles: 10
  systemReserved: {}
  kubeReserved: {}
  systemReservedCgroup: ""
  kubeReservedCgroup: ""
  enforceNodeAllocatable: ["pods"]
  EOF

  
  * address：kubelet 安全端口（https，10250）監聽的地址，不能為 127.0.0.1，否則 kube-apiserver、heapster 等不能調用 kubelet 的 API；
  * readOnlyPort=0：關閉只讀端口(默認 10255)，等效為未指定；
  * authentication.anonymous.enabled：設置為 false，不允許匿名?訪問 10250 端口；
  * authentication.x509.clientCAFile：指定簽名客戶端證書的 CA 證書，開啟 HTTP 證書認證；
  * authentication.webhook.enabled=true：開啟 HTTPs bearer token 認證；
  * 對于未通過 x509 證書和 webhook 認證的請求(kube-apiserver 或其他客戶端)，將被拒絕，提示 Unauthorized；
  * authroization.mode=Webhook：kubelet 使用 SubjectAcce***eview API 查詢 kube-apiserver 某 user、group 是否具有操作資源的權限(RBAC)；
  * featureGates.RotateKubeletClientCertificate、featureGates.RotateKubeletServerCertificate：自動 rotate 證書，證書的有效期取決于 kube-controller-manager 的 --experimental-cluster-signing-duration 參數；
  * 需要 root 賬戶運行；

2.4 創建kubelet 啟動服務

cat >/usr/lib/systemd/system/${KUBE_NAME}.service<<EOF
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service
[Service] 
WorkingDirectory=${K8S_INSTALL_PATH}
ExecStart=${K8S_BIN_PATH}/${KUBE_NAME} \\
  --bootstrap-kubeconfig=${KUBE_CONFIG_PATH}/kubelet-bootstrap.kubeconfig \\
  --kubeconfig=${KUBE_CONFIG_PATH}/kubelet.kubeconfig \\
  --config=${K8S_CONF_PATH}/kubelet-config.yaml \\
  --cert-dir=${CA_DIR} \\
  --hostname-override=${HOSTNAME} \\
  --pod-infra-container-image=registry.cn-beijing.aliyuncs.com/k8s_images/pause-amd64:3.1 \\
  --image-pull-progress-deadline=15m \\
  --cni-conf-dir=/etc/cni/net.d \\
  --container-runtime=docker \\
  --container-runtime-endpoint=unix:///var/run/dockershim.sock \\
  --root-dir=${K8S_INSTALL_PATH}/${KUBE_NAME} \\
  --volume-plugin-dir=${K8S_INSTALL_PATH}/${KUBE_NAME}/plugins \\
  --log-dir=${K8S_LOG_DIR}/${KUBE_NAME} \\
  --alsologtostderr=true \\
  --logtostderr=false \\
  --v=2
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF