016.Kubernetes二进制部署所有节点kube-proxy
2019-11-18 09:18:06来源:博客园 阅读 ()
016.Kubernetes二进制部署所有节点kube-proxy
一 部署 kube-proxy
kube-proxy 运行在所有节点上,它监听 apiserver 中 service 和 endpoint 的变化情况,创建路由规则以提供服务 IP 和负载均衡功能。1.1 安装kube-proxy
提示:k8smaster01节点已下载相应二进制,可直接分发至node节点。1.2 分发kube-proxy
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for all_ip in ${ALL_IPS[@]} 4 do 5 echo ">>> ${all_ip}" 6 scp kubernetes/server/bin/kube-proxy root@${all_ip}:/opt/k8s/bin/ 7 ssh root@${all_ip} "chmod +x /opt/k8s/bin/*" 8 done
1.3 创建kube-scheduler证书和私钥
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cat > kube-proxy-csr.json <<EOF 3 { 4 "CN": "system:kube-proxy", 5 "key": { 6 "algo": "rsa", 7 "size": 2048 8 }, 9 "names": [ 10 { 11 "C": "CN", 12 "ST": "Shanghai", 13 "L": "Shanghai", 14 "O": "k8s", 15 "OU": "System" 16 } 17 ] 18 } 19 EOF 20 #创建kube-scheduler的CA证书请求文件解释:
- CN:指定该证书的 User 为 system:kube-proxy;
- 预定义的 RoleBinding system:node-proxier 将User system:kube-proxy 与 Role system:node-proxier 绑定,该 Role 授予了调用 kube-apiserver Proxy 相关 API 的权限;
- 该证书只会被 kube-proxy 当做 client 证书使用,所以 hosts 字段为空。
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cfssl gencert -ca=/opt/k8s/work/ca.pem \ 3 -ca-key=/opt/k8s/work/ca-key.pem -config=/opt/k8s/work/ca-config.json \ 4 -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy #生成CA密钥(ca-key.pem)和证书(ca.pem)
1.4 创建和分发kubeconfig
kube-proxy 使用 kubeconfig 文件访问 apiserver,该文件提供了 apiserver 地址、嵌入的 CA 证书和 kube-proxy 证书:1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# kubectl config set-cluster kubernetes \ 4 --certificate-authority=/opt/k8s/work/ca.pem \ 5 --embed-certs=true \ 6 --server=${KUBE_APISERVER} \ 7 --kubeconfig=kube-proxy.kubeconfig 8 9 [root@k8smaster01 work]# kubectl config set-credentials kube-proxy \ 10 --client-certificate=kube-proxy.pem \ 11 --client-key=kube-proxy-key.pem \ 12 --embed-certs=true \ 13 --kubeconfig=kube-proxy.kubeconfig 14 15 [root@k8smaster01 work]# kubectl config set-context default \ 16 --cluster=kubernetes \ 17 --user=kube-proxy \ 18 --kubeconfig=kube-proxy.kubeconfig 19 20 [root@k8smaster01 work]# kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig 21 22 [root@k8smaster01 ~]# cd /opt/k8s/work 23 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 24 [root@k8smaster01 work]# for node_name in ${NODE_NAMES[@]} 25 do 26 echo ">>> ${node_name}" 27 scp kube-proxy.kubeconfig root@${node_name}:/etc/kubernetes/ 28 done
1.5 创建kube-proxy 配置文件
从 v1.10 开始,kube-proxy 部分参数可以配置文件中配置。可以使用 --write-config-to 选项生成该配置文件。1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cat > kube-proxy-config.yaml.template <<EOF 3 kind: KubeProxyConfiguration 4 apiVersion: kubeproxy.config.k8s.io/v1alpha1 5 clientConnection: 6 burst: 200 7 kubeconfig: "/etc/kubernetes/kube-proxy.kubeconfig" 8 qps: 100 9 bindAddress: ##ALL_IP## 10 healthzBindAddress: ##ALL_IP##:10256 11 metricsBindAddress: ##ALL_IP##:10249 12 enableProfiling: true 13 clusterCIDR: ${CLUSTER_CIDR} 14 hostnameOverride: ##ALL_NAME## 15 mode: "ipvs" 16 portRange: "" 17 kubeProxyIPTablesConfiguration: 18 masqueradeAll: false 19 kubeProxyIPVSConfiguration: 20 scheduler: rr 21 excludeCIDRs: [] 22 EOF解释:
- bindAddress: 监听地址;
- clientConnection.kubeconfig: 连接 apiserver 的 kubeconfig 文件;
- clusterCIDR: kube-proxy 根据 --cluster-cidr 判断集群内部和外部流量,指定 --cluster-cidr 或 --masquerade-all 选项后 kube-proxy 才会对访问 Service IP 的请求做 SNAT;
- hostnameOverride: 参数值必须与 kubelet 的值一致,否则 kube-proxy 启动后会找不到该 Node,从而不会创建任何 ipvs 规则;
- mode: 使用 ipvs 模式。
1.6 分发配置文件
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for (( i=0; i < 6; i++ )) 4 do 5 echo ">>> ${ALL_NAMES[i]}" 6 sed -e "s/##ALL_NAME##/${ALL_NAMES[i]}/" -e "s/##ALL_IP##/${ALL_IPS[i]}/" kube-proxy-config.yaml.template > kube-proxy-config-${ALL_NAMES[i]}.yaml.template 7 scp kube-proxy-config-${ALL_NAMES[i]}.yaml.template root@${ALL_NAMES[i]}:/etc/kubernetes/kube-proxy-config.yaml 8 done
1.7 创建kube-proxy的systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# cat > kube-proxy.service <<EOF 4 [Unit] 5 Description=Kubernetes Kube-Proxy Server 6 Documentation=https://github.com/GoogleCloudPlatform/kubernetes 7 After=network.target 8 9 [Service] 10 WorkingDirectory=${K8S_DIR}/kube-proxy 11 ExecStart=/opt/k8s/bin/kube-proxy \\ 12 --config=/etc/kubernetes/kube-proxy-config.yaml \\ 13 --logtostderr=true \\ 14 --v=2 15 Restart=on-failure 16 RestartSec=5 17 LimitNOFILE=65536 18 19 [Install] 20 WantedBy=multi-user.target 21 EOF
1.8 分发kube-proxy systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for all_name in ${ALL_NAMES[@]} 4 do 5 echo ">>> ${all_name}" 6 scp kube-proxy.service root@${all_name}:/etc/systemd/system/ 7 done #分发system
二 启动并验证
2.1 启动kube-proxy 服务
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for all_ip in ${ALL_IPS[@]} 4 do 5 echo ">>> ${all_ip}" 6 ssh root@${all_ip} "mkdir -p ${K8S_DIR}/kube-proxy" 7 ssh root@${all_ip} "modprobe ip_vs_rr" 8 ssh root@${all_ip} "systemctl daemon-reload && systemctl enable kube-proxy && systemctl restart kube-proxy" 9 done #启动服务前必须先创建工作目录
2.2 检查kube-proxy 服务
1 [root@k8smaster01 ~]# source /opt/k8s/bin/environment.sh 2 [root@k8smaster01 ~]# for all_ip in ${ALL_IPS[@]} 3 do 4 echo ">>> ${all_ip}" 5 ssh root@${all_ip} "systemctl status kube-proxy|grep Active" 6 done
2.3 查看监听端口
kube-proxy 监听 10249 和 10256 端口:- 10249:对外提供 /metrics;
- 10256:对外提供 /healthz 的访问。
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for all_ip in ${ALL_IPS[@]} 4 do 5 echo ">>> ${all_ip}" 6 ssh root@${all_ip} "sudo netstat -lnpt|grep kube-prox" 7 done
2.4 查看ipvs 路由规则
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for all_ip in ${ALL_IPS[@]} 4 do 5 echo ">>> ${all_ip}" 6 ssh root@${all_ip} "/usr/sbin/ipvsadm -ln" 7 done可见所有通过 https 访问 K8S SVC kubernetes 的请求都转发到 kube-apiserver 节点的 6443 端口。
原文链接:https://www.cnblogs.com/itzgr/p/11883391.html
如有疑问请与原作者联系
标签:
版权申明:本站文章部分自网络,如有侵权,请联系:west999com@outlook.com
特别注意:本站所有转载文章言论不代表本站观点,本站所提供的摄影照片,插画,设计作品,如需使用,请与原作者联系,版权归原作者所有
- 附020.Nginx-ingress部署及使用 2020-06-02
- corosync v1 + pacemaker高可用集群部署(二)资源配置(VIP+ 2020-05-28
- Nginx + Docker 多阶段构建的部署学习 2020-05-25
- 私有云技术 2020-05-23
- 运维相关 2020-05-16
IDC资讯: 主机资讯 注册资讯 托管资讯 vps资讯 网站建设
网站运营: 建站经验 策划盈利 搜索优化 网站推广 免费资源
网络编程: Asp.Net编程 Asp编程 Php编程 Xml编程 Access Mssql Mysql 其它
服务器技术: Web服务器 Ftp服务器 Mail服务器 Dns服务器 安全防护
软件技巧: 其它软件 Word Excel Powerpoint Ghost Vista QQ空间 QQ FlashGet 迅雷
网页制作: FrontPages Dreamweaver Javascript css photoshop fireworks Flash