云原生Kubernetes: K8S 1.26版本 部署KubeSphere
目录
一、实验
1.环境
2.K8S 1.26版本部署HELM
3.K8S 1.26版本 部署KubeSphere
4.安装KubeSphere DevOps
二、问题
1.如何安装Zadig
2.扩展插件Zadig安装失败
3.calico 如何实现不同node通信
4.如何清除docker占用的磁盘空间
5.如何强制删除资源
6.namespace删除不了
7.job如何实现删除资源
8.containerd容器使用ctr命令如何实现镜像操作
一、实验
1.环境
（1）主机
表1 主机
| 主机 | 架构 | 版本 | IP | 备注 |
| master1 | K8S master节点 | 1.26.0 | 192.168.204.190 | |
| node1 | K8S node节点 | 1.26.0 | 192.168.204.191 |
（2）Termius连接
（3）master节点查看集群
1）查看node
kubectl get node2)查看node详细信息
kubectl get node -o wide
2.K8S 1.26版本部署HELM
（1）查阅
https://github.com/helm/helm/tagsHELM版本与K8S集群兼容
（2）策略
当前K8S 集群为1.26.0版本，HELM 3.11.x 版本可以兼容。
所以选择3.11.0版本。
（3）下载
https://get.helm.sh/helm-v3.11.0-linux-amd64.tar.gzTermius使用SFTP传输
（4）解压
tar -zxvf helm-v3.11.0-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/helm
helm version
（5）命令补全
source <(helm completion bash)
3.K8S 1.26版本 部署KubeSphere
（1）查阅
https://docs.kubesphere.com.cn/v4.0/02-quickstart/01-install-ks-core/v1.26.x支持安装
（2）安装
helm upgrade --install -n kubesphere-system --create-namespace ks-core https://charts.kubesphere.io/main/ks-core-0.4.0.tgz
完整安装过程：
[root@master1 opt]# helm upgrade --install -n kubesphere-system --create-namespace ks-core https://charts.kubesphere.io/main/ks-core-0.4.0.tgz
Release "ks-core" does not exist. Installing it now.
NAME: ks-core
LAST DEPLOYED: Wed May 22 11:57:23 2024
NAMESPACE: kubesphere-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Please wait for several seconds for KubeSphere deployment to complete.1. Make sure KubeSphere components are running:kubectl get pods -n kubesphere-system2. Then you should be able to visit the console NodePort:Console: http://192.168.204.190:308803. To login to your KubeSphere console:Account: adminPassword: "P@88w0rd"NOTE: Please change the default password after login.For more details, please visit https://kubesphere.io.（3）查看pod
kubectl get pods -n kubesphere-system
(4) 访问
http://192.168.204.190:30880
（5）输入初始账户及密码
账户: admin
密码: P@88w0rd
（6）修改密码
（7）进入系统
(8) 集群管理
（9）扩展中心
（10）搜索市场
关键字“CI/CD”
4.安装KubeSphere DevOps
(1) 查阅
KubeSphere 扩展市场
https://kubesphere.com.cn/marketplace/extensions/devops/
其他方式安装
https://www.kubesphere.io/zh/docs/v3.4/quick-start/minimal-kubesphere-on-k8s/（2）同步云账户 
（3）安装
KubeSphere DevOps版本支持情况
Kubernetes 版本>=1.19.0-0KubeSphere 版本>=4.0.0-0
（4）下一步
（5）开始安装
安装中
成功
（6）下一步
（7）集群选择
（8）确认
（9）安装成功
（10）查看集群
二、问题
1.如何安装Zadig
(1) 查阅
Kubesphere扩展市场
https://kubesphere.com.cn/marketplace/extensions/zadig/
Zadig版本支持情况
Kubernetes 版本>=1.16.0-1.26.0KubeSphere 版本>=4.0.0-0
（2）Zadig主页
https://koderover.com/zadig（3）脚本方式安装
https://docs.koderover.com/zadig/Zadig%20v2.0.0/install/helm-deploy/（4）Kubesphere安装Zadig
官方脚本安装
curl -LO https://github.com/koderover/zadig/releases/download/v2.0.0/install_quickstart.sh
chmod +x ./install_quickstart.sh
（8）申明变量
export IP=192.168.204.190
export PORT=30090
（9）安装
./install_quickstart.sh这里安装过程预计持续 10 分钟左右
2.扩展插件Zadig安装失败
（1）报错
（2）原因分析
查看日志
2024-05-22T12:18:54.721997147+08:00 client.go:482: [debug] Ignoring delete failure for "zadig-post-upgrade" batch/v1, Kind=Job: jobs.batch "zadig-post-upgrade" not found
（3）解决方法
查阅相关问题
https://github.com/koderover/zadig/issues/2417先卸载
然后重新安装
下一步
开始安装
安装中
依然报错
2024-05-22T12:39:41.303302680+08:00 upgrade.go:442: [debug] warning: Upgrade "zadig" failed: post-upgrade hooks failed: 1 error occurred:
2024-05-22T12:39:41.303305360+08:00 * job failed: BackoffLimitExceeded
2024-05-22T12:39:41.303306374+08:00
2024-05-22T12:39:41.303307165+08:00
2024-05-22T12:39:41.327312921+08:00 Error: UPGRADE FAILED: post-upgrade hooks failed: 1 error occurred:
2024-05-22T12:39:41.327345989+08:00 * job failed: BackoffLimitExceeded
2024-05-22T12:39:41.327352567+08:00
2024-05-22T12:39:41.327356487+08:00
2024-05-22T12:39:41.327363940+08:00 helm.go:84: [debug] post-upgrade hooks failed: 1 error occurred:
2024-05-22T12:39:41.327368417+08:00 * job failed: BackoffLimitExceeded
2024-05-22T12:39:41.327371965+08:00
2024-05-22T12:39:41.327375169+08:00
2024-05-22T12:39:41.327380602+08:00 UPGRADE FAILED这时可以采用脚本安装。
查看脚本（HELM为3.6.1版本）与master1节点冲突
node1节点部署
如报错显示访问超时，需要网络好时再执行
Error: UPGRADE FAILED: post-upgrade hooks failed: timed out waiting for the condition
3.calico 如何实现不同node通信
（1）查看
calico网络插件
kubectl -n kube-system get po -owide | grep calico-node
可以看到有两个容器网段分别分配给了master1和node1
kubectl get ipamblocks
（2）查看具体信息
kubectl get ipamblocks 10-244-137-64-26 -oyaml
kubectl get ipamblocks 10-244-166-128-26 -oyaml
（3）查看路由表
查看master1路由表(目标地址为10.244.66.128/26的请求会被通过网卡tunl0转发到192.168.204.191，也就是node1上；master1节点本机上的POD IP，则会直接被路由到对应的calico网卡。)
route -n
查看node1路由表（node01上也可以看到类似的路由条目，目标地址为10.244.137.64/26的请求会被通过网卡tunl0转发到192.168.204.190，也就是master1上）
route -n
4.如何清除docker占用的磁盘空间
（1）查询
master1节点查看docker占用磁盘空间
docker system dfdocker system df -v
node1节点查看docker占用磁盘空间
（2）清除
master1节点仅删除停止运行的容器
docker container prune
noder1节点仅删除停止运行的容器
5.如何强制删除资源
（1）删除pod
命令
kubectl delete pod <your-pod-name> -n <name-space> --force --grace-period=0删除
kubectl delete --all pods -n zadig --force --grace-period=0
kubectl delete --all pods -n argocd --force --grace-period=0
kubectl delete --all pods -n extension-devops --force --grace-period=0
kubectl delete --all pods -n extension-zadig --force --grace-period=0
（2）删除pv、pvc
kubectl patch pv xxx -p '{"metadata":{"finalizers":null}}'
kubectl patch pvc xxx -p '{"metadata":{"finalizers":null}}'6.namespace删除不了
（1）报错
kubesphere-devops-system 一直为Terminating
（2）原因分析
选择一个Terminating namespace，并查看namespace 中的finalizer。
kubectl get namespace kubesphere-devops-system -o yaml
显示如下：
spec:finalizers:- kubernetes
（3）解决方法
导出json格式到文件
kubectl get namespace kubesphere-devops-system -o json >tmp.json
编辑tmp.josn，删除finalizers 字段的值
25 "spec": { #从此行开始删除26 "finalizers": [27 "kubernetes"28 ]29 }, # 删到此行
开启proxy（执行该命令后，当前终端会被卡住）
kubectl proxy
打开新的一个窗口，执行以下命令
curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/kubesphere-devops-system/finalize
确认处于Terminating 状态的namespace已经被删除
7.job如何实现删除资源
（1）查看状态
目前为已完成
（2）查看YAML
kind: Job
apiVersion: batch/v1
metadata:name: devops-post-deletenamespace: extension-devopslabels:controller-uid: 0e83e553-482f-4755-834a-9c0f07d4c5b9job-name: devops-post-deleteannotations:batch.kubernetes.io/job-tracking: ''helm.sh/hook: post-deletehelm.sh/hook-delete-policy: 'before-hook-creation,hook-succeeded'helm.sh/hook-weight: '1'revisions: >-{"1":{"status":"completed","succeed":1,"desire":1,"uid":"0e83e553-482f-4755-834a-9c0f07d4c5b9","start-time":"2024-05-22T15:52:49+08:00","completion-time":"2024-05-22T16:22:50+08:00"}}
spec:parallelism: 1completions: 1backoffLimit: 6selector:matchLabels:controller-uid: 0e83e553-482f-4755-834a-9c0f07d4c5b9template:metadata:creationTimestamp: nulllabels:controller-uid: 0e83e553-482f-4755-834a-9c0f07d4c5b9job-name: devops-post-deletespec:containers:- name: post-delete-jobimage: 'kubesphere/kubectl:v1.27.4'command:- /bin/bash- '-c'- |if kubectl get ns argocd; thenkubectl delete ns argocdfiif kubectl get ns kubesphere-devops-system; thenkubectl delete ns kubesphere-devops-systemfiif kubectl get ns kubesphere-devops-worker; thenkubectl delete ns kubesphere-devops-workerfiresources: {}terminationMessagePath: /dev/termination-logterminationMessagePolicy: FileimagePullPolicy: IfNotPresentrestartPolicy: NeverterminationGracePeriodSeconds: 30dnsPolicy: ClusterFirstsecurityContext: {}schedulerName: default-schedulercompletionMode: NonIndexedsuspend: false
8.containerd容器使用ctr命令如何实现镜像操作
（1）帮助命令
[root@node1 ~]# ctr images --help
NAME:ctr images - manage imagesUSAGE:ctr images command [command options] [arguments...]COMMANDS:check check existing images to ensure all content is available locallyexport export imagesimport import imageslist, ls list images known to containerdmount mount an image to a target pathunmount unmount the image from the targetpull pull an image from a remotepush push an image to a remotedelete, del, remove, rm remove one or more images by referencetag tag an imagelabel set and clear labels for an imageconvert convert an imageOPTIONS:--help, -h show help
（2）拉取
ctr images pull ghcr.io/dexidp/dex:v2.30.2
(3) 查看
crictl images list
或
ctr images list
或
ctr i ls -q
（4）导出
ctr image export dev.tar.gz ghcr.io/dexidp/dex:v2.30.2
（5）删除
1）查询
ctr image list | grep ghcr.io/dexidp/dex:v2.30.22）删除
ctr image delete ghcr.io/dexidp/dex:v2.30.2
（6）导入
1)导入
ctr image import dev.tar.gz2）查询
ctr image list | grep ghcr.io/dexidp/dex:v2.30.2
