openshift版本：openshift v3.6.173.0.5
使用oc(同kubectl)命令访问apiserver资源的时候，会使用到/root/.kube/config文件中使用的配置。
使用user访问apiserver
oc命令使用config中定义的user和证书(公钥和私钥)访问apiserver。使用如下命令查看当前使用的config上下文：monitor为当前的namespace，test-openshfit-com:8443为apiserver暴露的server，system:admin为访问apiserver使用的user名称
# oc config current-context monitor/test-openshfit-com:8443/system:admin
查看system:admin对应的证书(下面使用变量代替)
users: - name: system:admin/test-openshift-com:8443 user: client-certificate-data: ${CA} client-key-data: ${KEY}
导出证书，将下面decode出的内容分别保存到/home/ca.cert，/home/ca.key
# echo -n ${CA}|base64 --decode #/home/ca.cert
# echo -n ${KEY}|base64 --decode #/home/ca.key
使用如下方式即可访问cluster范围内的资源，该方式与oc命令的原理一样。下面以访问servers为例
APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}') curl $APISERVER/api/v1/services --cert /home/ca.cert --key /home/ca.key --user system:admin
使用serviceaccount访问apiserver
serviceaccount除了可以为pod提供secret外，还可以作为访问apiserver资源的凭证。使用如下命令创建一个名为curltest的serviceaccount，并获取其token
oc create serviceaccount curltest APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}') TOKEN=$(oc serviceaccounts get-token curltest)
使用如下命令进行rolebinding之后就可以查看namespaces为monitor下面的资源，但不可以查看其他namespace的资源。system:master可以看作一个超级账户，可参见user-facing-roles
oc policy add-role-to-user system:master -z curltest curl $APISERVER/api/v1/namespaces/monitor/services --header "Authorization: Bearer $TOKEN"
使用下面命令查看当前rolebinding情况，可以查看当前serveraccount可进行的操作权限。注：openshift的add-role-to-user/add-cluster-role-to-user其实就是kubernetes进行rolebinding/clusterrolebinding的操作，将一个role权限赋予一个user或serviceaccount。
# oc describe rolebinding system:master Name: system:master Namespace: monitor Created: 5 minutes ago Labels: <none> Annotations: <none> Role: /system:master Users: <none> Groups: <none> ServiceAccounts: curltest Subjects: <none> Verbs Non-Resource URLs Resource Names API Groups Resources [*] [] [] [*] [*] [*] [*] [] [] []
使用如下命令进行clusterrolebinding之后就可以访问cluster范围内的资源，首先需要删除先前的rolebinding
oc policy remove-role-from-user system:master -z curltest oadm policy add-cluster-role-to-user system:master -z curltest TOKEN=$(oc serviceaccounts get-token curltest) curl $APISERVER/api/v1/services --header "Authorization: Bearer $TOKEN"
查看clusterrolebinding情况
# oc describe clusterrolebinding system:master Name: system:masters Created: 2 weeks ago Labels: <none> Annotations: <none> Role: /system:master Users: <none> Groups: system:masters ServiceAccounts: monitor/curltest Subjects: <none> Verbs Non-Resource URLs Resource Names API Groups Resources [*] [] [] [*] [*] [*] [*] [] [] []
环境清理
oadm policy remove-cluster-role-from-user system:master -z liu oc delete sa curltest
下面演示pod如何使用serviceaccount访问apiserver资源，参照在Kubernetes Pod中使用Service Account访问API Server
首先安装minikube和go，方法可以参见https://www.cnblogs.com/charlieroro/p/10434138.html。minikube启动时直接使用docker驱动即可：minikube start --vm-driver=none
对client-go的操作步骤用于生成测试镜像，可以直接下载已经打包好的镜像(docker pull docker push woodliu268/k8s-example)来跳过下面相关操作
安装client-go，client使用了go module方式来管理包依赖(client-go根目录下使用go.mod和go.sum来管理包)，参见Installing client-go
export GO111MODULE=on go mod init go get k8s.io/client-go@master
修改client-go/examples/in-cluster-client-configuration/main.go目录下，将panic全部修改为fmt.Println，执行如下命令编译为可执行程序main
go build -o main main.go
Dockerfile内容如下，编译为docker镜像
FROM debian COPY main /root/main RUN chmod +x /root/main WORKDIR /root ENTRYPOINT ["/root/main"]
docker build -t k8s/example1:latest .
使用如下deployment创建pod，默认创建的default命名空间
# cat deployment.yaml apiVersion: extensions/v1beta1 kind: Deployment metadata: name: k8s-example spec: replicas: 1 template: metadata: labels: run: k8s-example spec: containers: - name: k8s-example image: k8s/example1:latest imagePullPolicy: IfNotPresent
kubectl log -f k8s-example-7747697dbf-772df时发现有如下错误。说明pod使用用户system:serviceaccount:default:default访问apiserver的时候访问失败
pods is forbidden: User "system:serviceaccount:default:default" cannot list resource "pods" in API group "" at the cluster scope There are 0 pods in the cluster
由于需要在cluster范围内访问pod资源，下面创建clusterrole和clusterrolebinding(参考Using RBAC Authorization)，并赋予system:serviceaccount:default:default list pod的权限
# cat clusterrole.yaml kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"]
# cat clusterrolebinding.yaml kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: system:serviceaccount:default:default apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: pod-reader apiGroup: rbac.authorization.k8s.io
重新创建deployment，查看pod日志，可以正常读取cluster的pod信息
There are 10 pods in the cluster
PS：
- 使用kubectl get RESOURECE -v=NUM可以查看kubectl的与apiserver的交互，RESOURECE为pod，service等；NUM取值为6-8
- 使用oc config use-context可以设置kubeconfig文件中的current-context字段
- Service account 验证时用户名 system:serviceaccount:(NAMESPACE):(SERVICEACCOUNT)，被指定到组 system:serviceaccounts 和 system:serviceaccounts:(NAMESPACE)
- 应用程序可能会在如yaml模板中使用serviceaccount挂载到pod中的tls证书来访问apiserver资源
参考：
https://kubernetes.io/docs/reference/access-authn-authz/rbac/
https://docs.openshift.com/container-platform/3.5/rest_api/index.html
https://docs.openshift.com/container-platform/3.9/admin_guide/manage_rbac.html
https://docs.openshift.com/enterprise/3.0/admin_guide/manage_authorization_policy.html
https://jimmysong.io/posts/user-authentication-in-kubernetes/