《k3s 源码解析3 ---- k3s集群搭建》-锐单电子商城

一、安装说明:

主机名

如果主机名称相同，要求主机名称不相同样品，可通过参数--with-node-id添加随机后缀，或者通过参数--node-name或环境变量$K3S_NODE_NAME指定主机名

操作系统基本上，你可以跑在所有的地方Linux在系统方面，官方支持和测试的系统包括

Ubuntu 16.04 (amd64)
Ubuntu 18.04 (amd64)
Raspbian Buster*

硬件资源

CPU：1
内存：512MB(建议至少1G)
磁盘：K3s性能取决于数据库的性能。建议运行SSD上

端口功能列表需要监控和开放的默认端口如下：

网络协议	端口	对应服务	功能描述
TCP	6443	K3s agent nodes	Kubernetes API
UCP	8472	K3s server and agent nodes	Required only for Flannel VXLAN
TCP	10250	K3s server and agent nodes	kubelet

指定网络插件 k3s 默认使用Flannel vxlan作为CNI可通过以下启动参数指定容器网络k3s网络插件类型：

CLI FLAG AND VALUE	DESCRIPTION
–flannel-backend=vxlan	使用vxlan(默认).
–flannel-backend=ipsec	使用IPSEC后端加密网络流量.
–flannel-backend=host-gw	使用host_gw模式.
–flannel-backend=wireguard	使用WireGuard后端加密网络流量。可能需要其他核心模块和配置.

若使用独立的CNI安装时可指定参数--flannel-backend=none, 然后单独安装自己的CNI

二、本实验节点(cpu异构环境：

ip	role
192.168.0.47 （x86节点）	k3s server (m8s master) node
192.168.0.16 (arm64 节点)	k3s agent (k8s worker) node

三、server安装：

1) 部署k3s:

1.1 默认数据库部署k3s server：

tsc@k8s-master:~$ sudo curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh - # 会作为一个systemd服务运行, 支持自启动启动 # 额外的工具将安装，包括kubectl, crictl, ctr, k3s-killall.sh和k3s-uninstall.sh # 会生成kubeconfig配置文件/etc/rancher/k3s/k3s.yaml  # 查看k3s server状态 tsc@k8s-master:~$ sudo systemctl status k3s  # 查看server token() tsc@k8s-master:~$ sudo cat /var/lib/rancher/k3s/server/node-token K10e222d41fa57ec6fb0ba020aa8633625728b672bc63a1a8c044f1cc45d9c9d489::server:efde08a2190c4622594abe0d3abeec41 tsc@k8s-master:~$  # 杀死k3s server/agent tsc@k8s-master:~$ sudo k3s-killall.sh

K3S_URL是Server节点的IP地址, 默认端口是6443
K3S_TOKEN存储在Server节点的/var/lib/rancher/k3s/server/node-token文件中

1.2 部署其他数据库k3s server:

# 使用MySQL数据库 curl -sfL https://get.k3s.io | sh -s - server \  --datastore-endpoint="mysql://username:password@tcp(hostname:3306)/database-name" # 使用PG数据库 curl -sfL https://get.k3s.io | sh -s - server \ --datastore-endpoint="postgres://username:password@hostname:port/database-name" # 使用etcd集群数据库 curl -sfL https://get.k3s.io | sh -s - server \ --datastore-endpoint="https://etcd-host-1:2379,https://etcd-host-2:2379,https://etcd-host-3:2379"

1.3 HA模式部署（内置数据库）：

这种模式下，Server节点个数必须是奇数个，推荐是三个Server节点。

1.3.1 启动第一个Server节点的时候需要携带--cluster-init参数，以及K3S_TOKEN

K3S_TOKEN=SECRET k3s server --cluster-init

1.3.2 然后在启动其他Server节点:

K3S_TOKEN=SECRET k3s server --server https://<ip or hostname of server1>:6443

2) 修改k3s service 启动参数(指定网卡)：

server节点是多网卡的情况下，需要指定使用哪个网卡（这里使用网卡：ens192），编辑server节点k3s服务（sudo vim /etc/systemd/system/k3s.service）：

[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target
After=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=/bin/sh -xc '! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service'
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s \
    server --flannel-iface ens192 \

3) 修改k3s server api端点：

vim /etc/rancher/k3s/k3s.yaml文件，将server api 从 https://127.0.0.1:6443修改为 https://192.168.0.47:6443:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkekNDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdGMyVnkKZG1WeUxXTmhRREUyTlRBM056TTFNak13SGhjTk1qSXdOREkwTURReE1qQXpXaGNOTXpJd05ESXhNRFF4TWpBegpXakFqTVNFd0h3WURWUVFEREJock0zTXRjMlZ5ZG1WeUxXTmhRREUyTlRBM056TTFNak13V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFUVXNNMVRIWDFPSUgya3duSFlmVGxmeUl4b0VQb0RJVDFrVzJHWjRCbGQKdlpwQys2cEVuZmNMTUZvaVI5VWYyTlF5K0F5OVpkZ201YnhtSnZ1bjUvaWRvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVUlOaklpTS85ZEhnQ0R5eTBPTGF5CmdvdW9qdjR3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnVDI0eUt4UGNTVDRueXFxZHBZR3RpVFlxdzZ1SXd0Yy8KeXpwb1VqRkxWSWtDSVFDMi8vS1hyKyttUEIrT1VzcFZpLys3TXdQMjVzZXlsQkx4aFc0T0ZJdFR3dz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    server: https://192.168.0.47:6443
  name: default
contexts:
- context:
    cluster: default
    user: default
  name: default
current-context: default
kind: Config
preferences: { 
        }
users:
- name: default
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJrakNDQVRlZ0F3SUJBZ0lJRElvMjdiMTRsK0F3Q2dZSUtvWkl6ajBFQXdJd0l6RWhNQjhHQTFVRUF3d1kKYXpOekxXTnNhV1Z1ZEMxallVQXhOalV3Tnpjek5USXpNQjRYRFRJeU1EUXlOREEwTVRJd00xb1hEVEl6TURReQpOREEwTVRJd00xb3dNREVYTUJVR0ExVUVDaE1PYzNsemRHVnRPbTFoYzNSbGNuTXhGVEFUQmdOVkJBTVRESE41CmMzUmxiVHBoWkcxcGJqQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VIQTBJQUJCYXRTRmN2RE85NXZkdFgKbG5DUk9nQ3huSTlubW5KL3U3UVI2MnlVK2lqL1ozVlBBd1NRdkYrMG10WDFod3hOeG9LS2ZyZGhMNTliQ0VrcwpCdFBhMm1DalNEQkdNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUtCZ2dyQmdFRkJRY0RBakFmCkJnTlZIU01FR0RBV2dCUTR5VnZhL25nMkNZUzJSTnp0ejRwbm1DR3BBREFLQmdncWhrak9QUVFEQWdOSkFEQkcKQWlFQTQwNGZJSFFhNnEvc3c2V01pdXZuNi9UbjNGZ3VhZVlqNDB5RHNCL2tnaFVDSVFEVWpoL3ZIRGFONThqbApxby9jWFpTUTJTd1ltMzBEWFQvVE1iSGVFbzhUbGc9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tQkVHSU4gQ0VSVElGSUNBVEUtLS0tLQpNSUlCZURDQ0FSMmdBd0lCQWdJQkFEQUtCZ2dxaGtqT1BRUURBakFqTVNFd0h3WURWUVFEREJock0zTXRZMnhwClpXNTBMV05oUURFMk5UQTNOek0xTWpNd0hoY05Nakl3TkRJME1EUXhNakF6V2hjTk16SXdOREl4TURReE1qQXoKV2pBak1TRXdId1lEVlFRRERCaHJNM010WTJ4cFpXNTBMV05oUURFMk5UQTNOek0xTWpNd1dUQVRCZ2NxaGtqTwpQUUlCQmdncWhrak9QUU1CQndOQ0FBVGxvUThYay92UW9hVVFRcFVJeGtZQ2tadlBIbnhPK0JCRkFUbisxOTVPCmVpK3c1Um0xSTIrdWgxK1dSb1JCT2U0bXFMYXRIZmZ5VzR2Y3kvMW9NN2NMbzBJd1FEQU9CZ05WSFE4QkFmOEUKQkFNQ0FxUXdEd1lEVlIwVEFRSC9CQVV3QXdFQi96QWRCZ05WSFE0RUZnUVVPTWxiMnY1NE5nbUV0a1RjN2MrSwpaNWdocVFBd0NnWUlLb1pJemowRUF3SURTUUF3UmdJaEFNOWNvOXQvT2lBM2MvNm5hZlNSWkQwbmtKYVlBYzhYCjhNdnBUUzJuVDc5SkFpRUFzaExaSkdNMGhqOUZ0c0hLVC9FeHZZK0Z1RE1Nbi85ajN3cFJpWFRCa05vPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    client-key-data: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUFWY2NOL3l1Q2RUUUFvaDUxc1lURkZwTTN5K1lqRHVXSlIvSytkT3NpVzFvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFRnExSVZ5OE03M205MjFlV2NKRTZBTEdjajJlYWNuKzd0QkhyYkpUNktQOW5kVThEQkpDOApYN1NhMWZXSERFM0dnb3ArdDJFdm4xc0lTU3dHMDlyYVlBPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=

重启k3s server:

 sudo systemctl daemon-reload
 sudo systemctl restart k3s

四、agent 安装，并加入k3s server集群：

root@k8s-node1:~# curl -sfL http://rancher-mirror.cnrancher.com/k3s/k3s-install.sh | K3S_URL=https://192.168.0.47:6443 K3S_TOKEN=K10e222d41fa57ec6fb0ba020aa8633625728b672bc63a1a8c044f1cc45d9c9d489::server:efde08a2190c4622594abe0d3abeec41 INSTALL_K3S_MIRROR=cn sh -

五、启动agent：

k3s agent --server https://192.168.0.47:6443 --token K10e222d41fa57ec6fb0ba020aa8633625728b672bc63a1a8c044f1cc45d9c9d489::server:efde08a2190c4622594abe0d3abeec41

六、查看k3s 集群节点：

tsc@k8s-master:~$ sudo kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml  get nodes -o wide
NAME         STATUS   ROLES                  AGE   VERSION        INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
k8s-master   Ready    control-plane,master   11h   v1.22.7+k3s1   192.168.0.46   <none>        Ubuntu 18.04.5 LTS   4.15.0-176-generic   containerd://1.5.9-k3s1
k8s-node1    Ready    <none>                 10h   v1.22.7+k3s1   192.168.0.16   <none>        Ubuntu 18.04.2 LTS   4.4.103              containerd://1.5.9-k3s1
tsc@k8s-master:~$

资讯详情

《k3s 源码解析3 ---- k3s集群搭建》