010.Kubernetes二进制部署kube-controller-manag…
2019-11-17 09:19:07来源:博客园 阅读 ()
010.Kubernetes二进制部署kube-controller-manager
一 部署高可用kube-controller-manager
1.1 高可用kube-controller-manager介绍
本实验部署一个三实例 kube-controller-manager 的集群,启动后将通过竞争选举机制产生一个 leader 节点,其它节点为阻塞状态。当 leader 节点不可用时,阻塞的节点将再次进行选举产生新的 leader 节点,从而保证服务的可用性。 为保证通信安全,本文档先生成 x509 证书和私钥,kube-controller-manager 在如下两种情况下使用该证书:- 与 kube-apiserver 的安全端口通信;
- 在安全端口(https,10252) 输出 prometheus 格式的 metrics。
1.2 创建kube-controller-manager证书和私钥
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cat > kube-controller-manager-csr.json <<EOF 3 { 4 "CN": "system:kube-controller-manager", 5 "hosts": [ 6 "127.0.0.1", 7 "172.24.8.71", 8 "172.24.8.72", 9 "172.24.8.73" 10 ], 11 "key": { 12 "algo": "rsa", 13 "size": 2048 14 }, 15 "names": [ 16 { 17 "C": "CN", 18 "ST": "Shanghai", 19 "L": "Shanghai", 20 "O": "system:kube-controller-manager", 21 "OU": "System" 22 } 23 ] 24 } 25 EOF 26 #创建kube-controller-manager的CA证书请求文件解释: hosts 列表包含所有 kube-controller-manager 节点 IP; CN 和 O 均为 system:kube-controller-manager,kubernetes 内置的 ClusterRoleBindings system:kube-controller-manager 赋予 kube-controller-manager 工作所需的权限。
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem \ 3 -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json \ 4 -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager #生成CA密钥(ca-key.pem)和证书(ca.pem)
1.3 分发证书和私钥
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 4 do 5 echo ">>> ${master_ip}" 6 scp kube-controller-manager*.pem root@${master_ip}:/etc/kubernetes/cert/ 7 done
1.4 创建和分发kubeconfig
kube-controller-manager 使用 kubeconfig 文件访问 apiserver,该文件提供了 apiserver 地址、嵌入的 CA 证书和 kube-controller-manager 证书:1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# kubectl config set-cluster kubernetes \ 4 --certificate-authority=/opt/k8s/work/ca.pem \ 5 --embed-certs=true \ 6 --server=${KUBE_APISERVER} \ 7 --kubeconfig=kube-controller-manager.kubeconfig 8 9 [root@k8smaster01 work]# kubectl config set-credentials system:kube-controller-manager \ 10 --client-certificate=kube-controller-manager.pem \ 11 --client-key=kube-controller-manager-key.pem \ 12 --embed-certs=true \ 13 --kubeconfig=kube-controller-manager.kubeconfig 14 15 [root@k8smaster01 work]# kubectl config set-context system:kube-controller-manager \ 16 --cluster=kubernetes \ 17 --user=system:kube-controller-manager \ 18 --kubeconfig=kube-controller-manager.kubeconfig 19 20 [root@k8smaster01 work]# kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig 21 22 [root@k8smaster01 ~]# cd /opt/k8s/work 23 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 24 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 25 do 26 echo ">>> ${master_ip}" 27 scp kube-controller-manager.kubeconfig root@${master_ip}:/etc/kubernetes/ 28 done
1.5 创建kube-controller-manager的systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# cat > kube-controller-manager.service.template <<EOF 4 [Unit] 5 Description=Kubernetes Controller Manager 6 Documentation=https://github.com/GoogleCloudPlatform/kubernetes 7 8 [Service] 9 WorkingDirectory=${K8S_DIR}/kube-controller-manager 10 ExecStart=/opt/k8s/bin/kube-controller-manager \\ 11 --profiling \\ 12 --cluster-name=kubernetes \\ 13 --controllers=*,bootstrapsigner,tokencleaner \\ 14 --kube-api-qps=1000 \\ 15 --kube-api-burst=2000 \\ 16 --leader-elect \\ 17 --use-service-account-credentials\\ 18 --concurrent-service-syncs=2 \\ 19 --bind-address=##MASTER_IP## \\ 20 --secure-port=10252 \\ 21 --tls-cert-file=/etc/kubernetes/cert/kube-controller-manager.pem \\ 22 --tls-private-key-file=/etc/kubernetes/cert/kube-controller-manager-key.pem \\ 23 --port=0 \\ 24 --authentication-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\ 25 --client-ca-file=/etc/kubernetes/cert/ca.pem \\ 26 --requestheader-allowed-names="" \\ 27 --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\ 28 --requestheader-extra-headers-prefix="X-Remote-Extra-" \\ 29 --requestheader-group-headers=X-Remote-Group \\ 30 --requestheader-username-headers=X-Remote-User \\ 31 --authorization-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\ 32 --cluster-signing-cert-file=/etc/kubernetes/cert/ca.pem \\ 33 --cluster-signing-key-file=/etc/kubernetes/cert/ca-key.pem \\ 34 --experimental-cluster-signing-duration=8760h \\ 35 --horizontal-pod-autoscaler-sync-period=10s \\ 36 --concurrent-deployment-syncs=10 \\ 37 --concurrent-gc-syncs=30 \\ 38 --node-cidr-mask-size=24 \\ 39 --service-cluster-ip-range=${SERVICE_CIDR} \\ 40 --pod-eviction-timeout=6m \\ 41 --terminated-pod-gc-threshold=10000 \\ 42 --root-ca-file=/etc/kubernetes/cert/ca.pem \\ 43 --service-account-private-key-file=/etc/kubernetes/cert/ca-key.pem \\ 44 --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\ 45 --logtostderr=true \\ 46 --v=2 47 Restart=on-failure 48 RestartSec=5 49 50 [Install] 51 WantedBy=multi-user.target 52 EOF
1.6 分发systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for (( i=0; i < 3; i++ )) 4 do 5 sed -e "s/##MASTER_NAME##/${MASTER_NAMES[i]}/" -e "s/##MASTER_IP##/${MASTER_IPS[i]}/" kube-controller-manager.service.template > kube-controller-manager-${MASTER_IPS[i]}.service 6 done #修正相应IP 7 [root@k8smaster01 work]# ls kube-controller-manager*.service 8 [root@k8smaster01 ~]# cd /opt/k8s/work 9 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 10 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 11 do 12 echo ">>> ${master_ip}" 13 scp kube-controller-manager-${master_ip}.service root@${master_ip}:/etc/systemd/system/kube-controller-manager.service 14 done #分发system
二 启动并验证
2.1 启动kube-controller-manager 服务
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 4 do 5 echo ">>> ${master_ip}" 6 ssh root@${master_ip} "mkdir -p ${K8S_DIR}/kube-controller-manager" 7 ssh root@${master_ip} "systemctl daemon-reload && systemctl enable kube-controller-manager && systemctl restart kube-controller-manager" 8 done
2.2 检查kube-controller-manager 服务
1 [root@k8smaster01 ~]# source /opt/k8s/bin/environment.sh 2 [root@k8smaster01 ~]# for master_ip in ${MASTER_IPS[@]} 3 do 4 echo ">>> ${master_ip}" 5 ssh root@${master_ip} "systemctl status kube-controller-manager|grep Active" 6 done
2.3 查看输出的 metrics
1 [root@k8smaster01 ~]# curl -s --cacert /opt/k8s/work/ca.pem --cert /opt/k8s/work/admin.pem --key /opt/k8s/work/admin-key.pem https://172.24.8.71:10252/metrics |head注意:以上命令在 kube-controller-manager 节点上执行。
2.4 查看权限
1 [root@k8smaster01 ~]# kubectl describe clusterrole system:kube-controller-manager
ClusteRole system:kube-controller-manager 的权限很小,只能创建 secret、serviceaccount 等资源对象,各 controller 的权限分散到 ClusterRole system:controller:XXX 中。
当在 kube-controller-manager 的启动参数中添加 --use-service-account-credentials=true 参数,这样 main controller 会为各 controller 创建对应的 ServiceAccount XXX-controller。内置的 ClusterRoleBinding system:controller:XXX 将赋予各 XXX-controller ServiceAccount 对应的 ClusterRole system:controller:XXX 权限。
1 [root@k8smaster01 ~]# kubectl get clusterrole|grep controller
如deployment controller:
1 [root@k8smaster01 ~]# kubectl describe clusterrole system:controller:deployment-controller
2.5 查看当前leader
1 [root@k8smaster01 ~]# kubectl get endpoints kube-controller-manager --namespace=kube-system -o yaml
kubelet 认证和授权:https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization
原文链接:https://www.cnblogs.com/itzgr/p/11876066.html
如有疑问请与原作者联系
标签:
版权申明:本站文章部分自网络,如有侵权,请联系:west999com@outlook.com
特别注意:本站所有转载文章言论不代表本站观点,本站所提供的摄影照片,插画,设计作品,如需使用,请与原作者联系,版权归原作者所有
上一篇:部署django
下一篇: 4. 系统信息-学习目标
- 附020.Nginx-ingress部署及使用 2020-06-02
- corosync v1 + pacemaker高可用集群部署(二)资源配置(VIP+ 2020-05-28
- Nginx + Docker 多阶段构建的部署学习 2020-05-25
- 私有云技术 2020-05-23
- 运维相关 2020-05-16
IDC资讯: 主机资讯 注册资讯 托管资讯 vps资讯 网站建设
网站运营: 建站经验 策划盈利 搜索优化 网站推广 免费资源
网络编程: Asp.Net编程 Asp编程 Php编程 Xml编程 Access Mssql Mysql 其它
服务器技术: Web服务器 Ftp服务器 Mail服务器 Dns服务器 安全防护
软件技巧: 其它软件 Word Excel Powerpoint Ghost Vista QQ空间 QQ FlashGet 迅雷
网页制作: FrontPages Dreamweaver Javascript css photoshop fireworks Flash