最近公司小伙伴在折腾k8s, 部署中遇到问题, 一番探究之下原来是k8s 1.24版本后不再使用docker作为默认的容器平台, 而是提供统一的容器运行时接口CRI。 之前在公司和客户部署的都是k8s 1.16版本, 部署方式有所不同，遂记录下。

系统要求

  • 阿里云ECS 抢占实例，定时释放版
  • OS CentOS 7.9 x86
  • docker 20.10.17
  • containerd 1.6.6
  • kubernets 1.24.2

docker部署

  • 一键安装
curl -sSL https://get.daocloud.io/docker | sh
  • 设置开机启动
systemctl start docker && systemctl enable docker
  • 将driver由cgroupfs改为syetmd
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://0sfv2fhl.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

配置containerd

  • 生成默认配置文件
containerd config default > /etc/containerd/config.toml
  • 将 sandbox_image 镜像源设置为阿里云 google_containers 镜像源
vim /etc/containerd/config.toml
45  [plugins."io.containerd.grpc.v1.cri"]
61     sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
  • 配置镜像加速地址
#配置containerd镜像加速地址
vim /etc/containerd/config.toml
217  [plugins."io.containerd.tracing.processor.v1.otlp"]
218    endpoint = "https://0sfv2fhl.mirror.aliyuncs.com"
219    insecure = false
220    protocol = ""
#使用的是镜像加速器服务,阿里云镜像站开通
  • 配置 cgroup 驱动程序systemd
vim /etc/containerd/config.toml
114 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
125   SystemdCgroup = true
  • 应用所有更改后,重新启动containerd
systemctl restart containerd && systemctl enable containerd

netstat -anput | grep containerd

k8s部署

  • 允许 iptables 检查桥接流量
sudo modprobe br_netfilter

sudo lsmod | grep br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
  • 检查所需端口
yum install -y nc

nc 127.0.0.1 6443
  • 禁用swap分区
# 暂时关闭SWAP分区
swapoff -a

# 永久禁用SWAP分区
swapoff -a && sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
  • 添加阿里云k8s repo
vim /etc/yum.repos.d/kubernetes.repo

# 在这个文件中输入
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enable=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

# 将 SELinux 设置为 permissive 模式（相当于将其禁用）
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

sudo systemctl enable --now kubelet
  • 初始化集群
kubeadm init --image-repository registry.aliyuncs.com/google_containers --v=5
  • 配置默认的endpoints
crictl config runtime-endpoint unix:///run/containerd/containerd.sock
crictl config image-endpoint unix:///run/containerd/containerd.sock
  • 更新用户配置文件
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
  • flannel网络部署
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

kubectl apply -f kube-flannel.yml

稍等片刻, 便大功告成! 😄

NAME     STATUS   ROLES           AGE     VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
master   Ready    control-plane   4m27s   v1.24.2   172.18.54.227   <none>        CentOS Linux 7 (Core)   3.10.0-1160.66.1.el7.x86_64   containerd://1.6.6
Table of Contents