kubeadm 安装kubernetes 1.18.2版本 及踩坑

操作系统: CentOS Linux release 7.8.2003

三台虚拟机: 192.168.1.192 nj-idc-sit-zddtest0001 master 192.168.1.193 nj-idc-sit-zddtest0002 node 192.168.1.194 nj-idc-sit-zddtest0003 node

1-13步需要在3个节点上都执行

1、停止所有机器 firewalld 防火墙

systemctl stop firewalld
systemctl disable firewalld

2、关闭 swap

swapoff -a 
sed -i 's/.*swap.*/#&/' /etc/fstab

3、关闭 Selinux

setenforce  0 
sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/sysconfig/selinux 
sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config 
sed -i "s/^SELINUX=permissive/SELINUX=disabled/g" /etc/sysconfig/selinux 
sed -i "s/^SELINUX=permissive/SELINUX=disabled/g" /etc/selinux/config

4、没有启用DNS,配置hosts

cat << EOF >> /etc/hosts
192.168.1.192 nj-idc-sit-zddtest0001
192.168.1.193 nj-idc-sit-zddtest0002
192.168.1.194 nj-idc-sit-zddtest0003
EOF

5、安装环境依赖

yum install -y nfs-utils curl yum-utils device-mapper-persistent-data lvm2 net-tools conntrack-tools wget vim  ntpdate libseccomp libtool-ltdl telnet

6、升级内核版本解决Docker-ce版本兼容问题

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org && \
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm && \
yum --disablerepo=\* --enablerepo=elrepo-kernel repolist && \
yum --disablerepo=\* --enablerepo=elrepo-kernel install -y kernel-ml.x86_64 && \
yum remove -y kernel-tools-libs.x86_64 kernel-tools.x86_64 && \
yum --disablerepo=\* --enablerepo=elrepo-kernel install -y kernel-ml-tools.x86_64 && \
grub2-set-default 0
modprobe br_netfilter

重启

reboot

查看内核版本

uname -r

5.7.4-1.el7.elrepo.x86_64

7、调整内核参数

cat <<EOF >  /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.ipv4.tcp_tw_recycle = 0
vm.swappiness = 0
vm.overcommit_memory = 1
vm.panic_on_oom = 0
fs.inotify.max_user_instances = 8192
fs.inotify.max_user_watches = 1048576
fs.file-max = 52706963
fs.nr_open = 52706963
net.ipv6.conf.all.disable_ipv6 = 1
net.netfilter.nf_conntrack_max = 2310720
EOF
sysctl -p /etc/sysctl.d/k8s.conf
ls /proc/sys/net/bridge

8、调整系统时区

timedatectl set-timezone Asia/Shanghai
timedatectl set-local-rtc 0
systemctl restart rsyslog
systemctl restart crond
ntpdate -u ntp.api.bz

9、设置日志保存方式, 使用systemd journald

mkdir /var/log/journal
mkdir /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF
[Journal]
#持久化保存到磁盘
Storage=persistent

#压缩历史日志
Compress=yes

SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000

#最大占用空间
SystemMaxUse=10G

#单日志文件最大
SystemMaxFileSize=200M

#日志保存时间2周
MaxRetentionSec=2week

#不将日志转发到 syslog
ForwardToSyslog=no
EOF

systemctl restart systemd-journald

10、kube-proxy开启IPVS到前置条件

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF

chmod 755 /etc/sysconfig/modules/ipvs.modules && bash
/etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

11、安装docker

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager \
  --add-repo \
  http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum update -y && yum install -y docker-ce

mkdir /etc/docker

#配置daemon
cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  }
}
EOF
mkdir -p /etc/systemd/system/docker.service.d

systemctl daemon-reload && systemctl restart docker && systemctl enable docker

12、安装kubeadm, kubelet, kubectl

cat <<EOF > /etc/yum.repos.d/kubernetes.repo 
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg 
https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF


yum -y install kubeadm-1.18.2 kubelet-1.18.2 kubectl-1.18.2 --setopt=obsoletes=0

systemctl enable kubelet.service

13、下载镜像

docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.18.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.18.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.18.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.18.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.3-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.7

docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.18.2 k8s.gcr.io/kube-apiserver:v1.18.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.18.2 k8s.gcr.io/kube-controller-manager:v1.18.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.18.2 k8s.gcr.io/kube-scheduler:v1.18.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.18.2 k8s.gcr.io/kube-proxy:v1.18.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 k8s.gcr.io/pause:3.2
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.3-0 k8s.gcr.io/etcd:3.4.3-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.6.7 k8s.gcr.io/coredns:1.6.7

14、初始化主节点(在master节点上执行)

输出kubeadm默认到初始化配置

kubeadm config print init-defaults > kubeadm-config.yml

修改 advertiseAddress: 本机IP 修改 kubernetesVersion: v1.18.2 networking 下面添加 podSubnet: “10.244.0.0/16”

末尾添加

apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration featureGates: SupportIPVSProxyMode: true mode: ipvs

kubeadm init --config=kubeadm-config.yml |tee kubeadm-init.log

日志: [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.1.192:6443 –token abcdef.0123456789abcdef
–discovery-token-ca-cert-hash sha256:953f5f5de1c9418263fa2ed161d4e74e6a70341b4daa0229460b3adfe2e7635a

配置kubectl使用的配置文件

mkdir /root/.kube
cp -i /etc/kubernetes/admin.conf .kube/config

查看

kubectl get nodes

NAME STATUS ROLES AGE VERSION nj-idc-sit-zddtest0001 NotReady master 6m42s v1.18.2 现在节点状态是NotReady

15、部署网络flannel (在master节点上执行)

wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

kubectl apply -f kube-flannel.yml

查看pod

kubectl get pods -n kube-system

NAME READY STATUS RESTARTS AGE coredns-66bff467f8-2llpq 11 Running 0 17m coredns-66bff467f8-kksr7 11 Running 0 17m etcd-nj-idc-sit-zddtest0001 11 Running 0 17m kube-apiserver-nj-idc-sit-zddtest0001 11 Running 0 17m kube-controller-manager-nj-idc-sit-zddtest0001 11 Running 0 17m kube-flannel-ds-amd64-6l2r6 11 Running 0 80s kube-proxy-d8n2p 11 Running 0 17m kube-scheduler-nj-idc-sit-zddtest0001 11 Running 0 17m

查看node

kubectl get nodes

NAME STATUS ROLES AGE VERSION nj-idc-sit-zddtest0001 Ready master 16m v1.18.2

网络打通后,节点状态变成了Ready

16、现在可以将其他节点加入到k8s中(在node节点上执行)

通过命令

kubeadm join 192.168.1.192:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:953f5f5de1c9418263fa2ed161d4e74e6a70341b4daa0229460b3adfe2e7635a

查看pod

kubectl get pods -n kube-system

NAME READY STATUS RESTARTS AGE coredns-66bff467f8-2llpq 11 Running 0 157m coredns-66bff467f8-kksr7 11 Running 0 157m etcd-nj-idc-sit-zddtest0001 11 Running 0 157m kube-apiserver-nj-idc-sit-zddtest0001 11 Running 0 157m kube-controller-manager-nj-idc-sit-zddtest0001 11 Running 0 157m kube-flannel-ds-amd64-6l2r6 11 Running 0 141m kube-flannel-ds-amd64-bptkj 11 Running 0 57m kube-flannel-ds-amd64-fgqzs 11 Running 0 57m kube-proxy-6zdt2 11 Running 0 57m kube-proxy-d8n2p 11 Running 0 157m kube-proxy-h585l 11 Running 0 57m kube-scheduler-nj-idc-sit-zddtest0001 11 Running 0 157m

查看node

kubectl get nodes

NAME STATUS ROLES AGE VERSION nj-idc-sit-zddtest0001 Ready master 158m v1.18.2 nj-idc-sit-zddtest0002 Ready 57m v1.18.2 nj-idc-sit-zddtest0003 Ready 57m v1.18.2

17、踩坑

执行命令yum -y install kubeadm-1.18.2 kubelet-1.18.2 kubectl-1.18.2 时报下面的错误:

错误:软件包:kubelet-1.18.2-0.x86_64 (kubernetes)
          需要:kubernetes-cni = 0.6.0
          可用: kubernetes-cni-0.3.0.1-0.07a8a2.x86_64 (kubernetes)
              kubernetes-cni = 0.3.0.1-0.07a8a2
          可用: kubernetes-cni-0.5.1-0.x86_64 (kubernetes)
              kubernetes-cni = 0.5.1-0
          可用: kubernetes-cni-0.5.1-1.x86_64 (kubernetes)
              kubernetes-cni = 0.5.1-1
          可用: kubernetes-cni-0.6.0-0.x86_64 (kubernetes)
              kubernetes-cni = 0.6.0-0
          正在安装: kubernetes-cni-0.7.5-0.x86_64 (kubernetes)
              kubernetes-cni = 0.7.5-0
 您可以尝试添加 --skip-broken 选项来解决该问题
 您可以尝试执行:rpm -Va --nofiles --nodigest

解决:

加上参数–setopt=obsoletes=0

分享 提问
comments powered by Disqus