k3s部署及应用

2026-05-09-k3s部署及应用

1. k3s的部署及应用

1.0 k8s的架构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Control Plane (Master)
├── kube-apiserver (独立进程)
├── etcd (独立进程)
├── kube-controller-manager (独立进程)
├── kube-scheduler (独立进程)
└── 其他

Worker Node
├── kubelet
├── kube-proxy
├── containerd (独立安装)
└── 需要单独装 CNI 插件(Calico/Flannel)
└── 需要单独装 Ingress Controller(Nginx)
└── 需要单独装存储方案

1.1 k3s的架构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Server (相当于 master)
└── k3s 单个二进制 (~50MB)
    └── 内嵌启动所有组件:
        ├── kube-apiserver
        ├── kube-controller-manager
        ├── kube-scheduler
        ├── kubelet
        ├── kube-proxy
        ├── 数据存储 (默认 SQLite,可换 etcd/MySQL/PostgreSQL)
        ├── 容器运行时 (containerd,内嵌)
        ├── CNI (Flannel,内嵌)
        ├── Ingress Controller (Traefik,内嵌)
        ├── LoadBalancer (Klipper,内嵌)
        ├── Local Path Provisioner (存储,内嵌)
        ├── Metrics Server (内嵌)
        └── CoreDNS (内嵌)

# 查看 Traefik 版本和配置
kubectl get helmchart traefik -n kube-system -o yaml 

Agent (相当于 worker)
└── k3s agent 单个二进制
    └── 内嵌启动:
        ├── kubelet
        ├── kube-proxy
        └── containerd (内嵌)

1.1 k3s的介绍

k3s 是一个基于k8s的轻量级容器编排平台，它在k8s的基础上，添加了一些额外的功能，比如自动发现、自动扩展、自动回滚等。

1.2 k3s的部署

系统类型 IP 节点 Hostname
Rocky Linux release 9.7 192.168.56.132 master k3s-master01
Rocky Linux release 9.7 192.168.56.133 worker k3s-node01
Rocky Linux release 9.7 192.168.56.134 worker k3s-node02 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# 前置步骤
-------------------------------------
# 1. 设置主机名
yum install -y wget curl vim net-tools bash-completion
# master 节点执行
hostnamectl set-hostname k3s-master

# node01 执行
hostnamectl set-hostname k3s-node01

# node02 执行
hostnamectl set-hostname k3s-node02
-------------------------------------
# 2. 配置hosts文件
cat >> /etc/hosts <<EOF
192.168.232.10 k3s-master
192.168.232.11 k3s-node01
192.168.232.12 k3s-node02
EOF
-------------------------------------
# 3. 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
-------------------------------------
# 4. 关闭SELinux
# 临时关闭
setenforce 0

# 永久关闭
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config

# 验证
getenforce
# 输出 Permissive 就对
-------------------------------------
# 5. 关闭swap
# 临时关闭
swapoff -a

# 永久关闭
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

# 验证
free -h
# Swap 行应该全是 0
-------------------------------------
# 6. 安装 chrony
yum install -y chrony

# 启动
systemctl enable --now chronyd

# 验证
chronyc tracking
timedatectl
-------------------------------------
# 7. 配置内核参数
cat > /etc/sysctl.d/k3s.conf <<EOF
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

sysctl --system

# 验证
sysctl net.ipv4.ip_forward
# 输出 net.ipv4.ip_forward = 1
-------------------------------------
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
-------------------------------------
# 1. 配置镜像加速
[daocloud](https://github.com/DaoCloud/public-image-mirror)
cat >>/etc/rancher/k3s/registry.yaml <<EOF
mirrors:
  docker.io:
    endpoint:
      - "https://docker.m.daocloud.io"
  registry.k8s.io:
    endpoint:
      - "https://k8s.m.daocloud.io"
  quay.io:
    endpoint:
      - "https://quay.m.daocloud.io"
  gcr.io:
    endpoint:
      - "https://gcr.m.daocloud.io"
EOF
---------------------------------------
curl -sfL https://get.k3s.io | sh -
# 或者
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | \
  INSTALL_K3S_MIRROR=cn \
  K3S_TOKEN=k3s@Secret2026 \
  sh -s - server \
  --cluster-init \
  --write-kubeconfig-mode 644 \
  --node-ip=192.168.56.132
---------------------------------------

1.3 k3s的验证

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
-------------------------------------
# 1. 看服务状态
systemctl status k3s

# 看节点(等 30 秒)
kubectl get nodes

# 看系统 Pod
kubectl get pods -A

# 输出示例:
# NAME                  STATUS   ROLES                       AGE
# k3s-master            Ready    control-plane,etcd,master   1m
-------------------------------------
# 2. 配置kubectl补全
# 生成补全脚本
kubectl completion bash > /etc/bash_completion.d/kubectl

# 立即生效
source /etc/bash_completion.d/kubectl

# 或加到 bashrc
echo 'source <(kubectl completion bash)' >> ~/.bashrc
echo 'alias k=kubectl' >> ~/.bashrc
echo 'complete -F __start_kubectl k' >> ~/.bashrc
source ~/.bashrc
-------------------------------------
# 3. 查 Token
cat /var/lib/rancher/k3s/server/node-token

# 查 master IP
ip addr show | grep -E "inet.*eth|inet.*ens" | awk '{print $2}'
-------------------------------------
# 4. 部署 node01 node02
# 执行在 node01 节点上
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | \
  INSTALL_K3S_MIRROR=cn \
  K3S_URL=https://192.168.56.132:6443 \
  K3S_TOKEN=k3s@Secret2026 \
  sh -s - agent \
  --node-ip=192.168.56.133 # node02 改为 192.168.56.134
-------------------------------------

1.4 k3s优化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# 1. 配置节点角色
# 执行在 master 节点上
# 打 worker 角色标签(ROLES 列显示用)
kubectl label nodes k3s-node01 node-role.kubernetes.io/worker=
kubectl label nodes k3s-node02 node-role.kubernetes.io/worker=

# 打调度标签(后面 Prometheus、Grafana 用)
kubectl label nodes k3s-node01 workload=apps
kubectl label nodes k3s-node02 workload=apps

# 验证
kubectl get nodes
# NAME          STATUS   ROLES                       AGE
# k3s-master    Ready    control-plane,etcd,master
# k3s-node01    Ready    worker
# k3s-node02    Ready    worker
-------------------------------------
# 2. 配置普通用户使用 kubectl
# root 用户直接用 kubectl 就行
# 如果有普通用户要用:
mkdir -p ~/.kube
cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

# 如果 IP 是 127.0.0.1 要改成实际 IP
sed -i 's/127.0.0.1/192.168.56.132/g' ~/.kube/config

2. k3s的应用

2.1 headlamp

2.1.1 介绍headlamp

headlamp 是一个基于k8s的监控平台，它可以帮助用户快速地部署、升级和管理k8s应用。

2.1.2 部署headlamp

1
2
3
4
5
6
7
8
9
helm repo add headlamp https://kubernetes-sigs.github.io/headlamp/
helm install my-headlamp headlamp/headlamp \
    --namespace headlamp \
    --create-namespace \
    --set service.type=NodePort \
    --set service.nodePort=30808

# Helm 安装应用时，所有的行为都由 values.yaml 决定。默认配置：Headlamp 官方 Chart 默认的 service.type 通常就是 ClusterIP
kubectl patch svc my-headlamp -n headlamp -p '{"spec": {"type": "NodePort"}}'

2.1.3 访问headlamp

1
2
3
http://192.168.56.132:31049/
kubectl create token headlamp -n headlamp
kubectl get svc my-headlamp -n headlamp

2.2 prometheus

2.2.1 介绍prometheus

prometheus 是一个基于时间序列的监控系统，它可以帮助用户监控应用的运行状态、性能指标等。

2.2.2 部署prometheus

1
2
3
4
5
6
7
8
kubectl create namespace monitoring

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts/
helm repo update

helm install prometheus prometheus-community/prometheus \
  -f prometheus-values.yaml \
  -n monitoring

2.2.3 配置prometheus

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
# prometheus-values.yaml
cat > prometheus-values.yaml <<EOF
server:
  image:
    repository: docker.m.daocloud.io/prom/prometheus
    tag: v2.53.1

  nodeSelector:
    workload: apps

  resources:
    requests:
      memory: 256Mi
      cpu: 100m
    limits:
      memory: 512Mi
      cpu: 500m

  retention: "7d"

  persistentVolume:
    enabled: false

  service:
    type: NodePort
    nodePort: 30090

alertmanager:
  enabled: false

prometheus-pushgateway:
  enabled: false

prometheus-node-exporter:
  image:
    registry: docker.m.daocloud.io
    repository: prom/node-exporter
  resources:
    requests:
      memory: 32Mi
      cpu: 50m
    limits:
      memory: 64Mi
      cpu: 100m

kube-state-metrics:
  image:
    registry: k8s.m.daocloud.io
    repository: kube-state-metrics/kube-state-metrics
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 64Mi
      cpu: 50m
    limits:
      memory: 128Mi
      cpu: 200m

configmapReload:
  prometheus:
    image:
      repository: docker.m.daocloud.io/jimmidyson/configmap-reload
    nodeSelector:
      workload: apps
EOF

2.2.3 访问prometheus

1
2
http://192.168.56.132:30090/
kubectl get svc prometheus

2.3 grafana

2.3.1 介绍grafana

grafana 是一个基于k8s的监控平台，它可以帮助用户监控应用的运行状态、性能指标等。

2.3.2 部署grafana

1
2
3
4
5
6
helm repo add grafana https://grafana.github.io/helm-charts/
helm repo update

helm install grafana grafana/grafana \
  -f grafana-values.yaml \
  -n monitoring

2.3.3 配置grafana

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
# grafana-values.yaml
cat > grafana-values.yaml <<EOF
image:
  repository: docker.m.daocloud.io/grafana/grafana
  tag: "10.4.2"

nodeSelector:
  workload: apps

resources:
  requests:
    memory: 150Mi
    cpu: 50m
  limits:
    memory: 400Mi
    cpu: 300m

persistence:
  enabled: false

adminPassword: admin123

service:
  type: NodePort
  nodePort: 30030

datasources:
  datasources.yaml:
    apiVersion: 1
    datasources:
    - name: Prometheus
      type: prometheus
      url: http://prometheus-server.monitoring.svc.cluster.local
      access: proxy
      isDefault: true

sidecar:
  image:
    repository: docker.m.daocloud.io/kiwigrid/k8s-sidecar
  datasources:
    enabled: true
  dashboards:
    enabled: true

initChownData:
  enabled: false

dashboardProviders:
  dashboardproviders.yaml:
    apiVersion: 1
    providers:
    - name: 'default'
      orgId: 1
      folder: ''
      type: file
      disableDeletion: false
      editable: true
      options:
        path: /var/lib/grafana/dashboards/default

dashboards:
  default:
    k8s-cluster:
      gnetId: 7249
      revision: 1
      datasource: Prometheus
    node-exporter:
      gnetId: 1860
      revision: 37
      datasource: Prometheus
EOF

2.3.3 访问grafana

1
2
http://192.168.56.132:30030/
kubectl get svc grafana

2.4 Harbor 镜像仓库

2.4.1 介绍harbor

harbor 是一个基于k8s的镜像仓库,它可以帮助用户存储、分发和管理镜像。

生成证书 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
# 生成证书
自签证书 ca.crt 安装位置：
├── 本机系统证书库    → 让 Docker、curl 等工具信任
├── 本机浏览器        → 让浏览器信任（需要手动导入）
└── K3s/K8s 节点     → 让 containerd 拉镜像不报错

# 创建规范目录
mkdir -p /etc/harbor/certs
chmod 755 /etc/harbor/certs

# 生成证书到规范目录
cd /etc/harbor/certs

# CA 证书
openssl genrsa -out ca.key 4096
openssl req -x509 -new -nodes -sha512 -days 3650 \
  -subj "/CN=Harbor CA" \
  -key ca.key -out ca.crt

# 服务端证书
openssl genrsa -out harbor.key 4096
openssl req -sha512 -new \
  -subj "/CN=harbor.k3s.local" \
  -key harbor.key \
  -out harbor.csr

cat > v3.ext <<EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = harbor.k3s.local
DNS.2 = harbor.k8s.local
IP.1 = 192.168.56.132
IP.2 = 192.168.56.133
IP.3 = 192.168.56.134
IP.4 = 127.0.0.1
EOF

openssl x509 -req -sha512 -days 3650 \
  -extfile v3.ext \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -in harbor.csr -out harbor.crt

# 设置规范权限
chmod 600 ca.key harbor.key harbor.csr  # 私钥只有 root 可读
chmod 644 ca.crt harbor.crt v3.ext     # 证书公开可读
chown root:root /etc/harbor/certs/*

echo "证书生成完成"
ls -la /etc/harbor/certs/

信任证书 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# 每个节点都需要
# 创建目录
mkdir -p /etc/containerd/certs.d/harbor.k3s.local

# 复制 CA 证书
cp /etc/harbor/certs/ca.crt \
  /etc/containerd/certs.d/harbor.k3s.local/ca.crt

# 设置权限（containerd 需要能读）
chmod 644 /etc/containerd/certs.d/harbor.k3s.local/ca.crt
chown root:root /etc/containerd/certs.d/harbor.k3s.local/ca.crt

# hosts.toml
cat > /etc/containerd/certs.d/harbor.k3s.local/hosts.toml <<'EOF'
server = "https://harbor.k3s.local"

[host."https://harbor.k3s.local"]
  capabilities = ["pull", "resolve", "push"]
  ca = "/etc/containerd/certs.d/harbor.k3s.local/ca.crt"
EOF

chmod 644 /etc/containerd/certs.d/harbor.k3s.local/hosts.toml

# docker
mkdir -p /etc/docker/certs.d/harbor.k3s.local

cp /etc/harbor/certs/ca.crt \
  /etc/docker/certs.d/harbor.k3s.local/ca.crt

chmod 644 /etc/docker/certs.d/harbor.k3s.local/ca.crt
chown root:root /etc/docker/certs.d/harbor.k3s.local/ca.crt

# 信任 CA 根证书
cp /etc/harbor/certs/ca.crt \
  /etc/pki/ca-trust/source/anchors/harbor-ca.crt
update-ca-trust

# 验证
openssl verify -CAfile /etc/harbor/certs/ca.crt /etc/harbor/certs/harbor.crt

2.4.2 部署harbor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# 添加 harbor 仓库
helm repo add harbor https://helm.goharbor.io
helm repo update

# 创建 namespace
kubectl create namespace harbor

# 创建 TLS 证书
kubectl create secret tls harbor-tls \
  --cert=/etc/harbor/certs/harbor.crt \   # 服务端证书
  --key=/etc/harbor/certs/harbor.key \    # 服务端私钥
  -n harbor

# 保存默认 values 作参考
helm show values harbor/harbor > harbor-default-values.yaml

helm install harbor harbor/harbor \
  -f harbor-https-values-k3s-mkcert.yaml \
  -n harbor \
  --timeout 10m

# 验证
kubectl get svc -n harbor
[root@k3s-master01 certs.d]# kubectl get svc -n harbor
NAME                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
harbor-core         ClusterIP   10.43.123.104   <none>        80/TCP              44m
harbor-database     ClusterIP   10.43.163.202   <none>        5432/TCP            44m
harbor-jobservice   ClusterIP   10.43.204.210   <none>        80/TCP              44m
harbor-portal       ClusterIP   10.43.68.134    <none>        80/TCP              44m
harbor-redis        ClusterIP   10.43.169.29    <none>        6379/TCP            44m
harbor-registry     ClusterIP   10.43.141.88    <none>        5000/TCP,8080/TCP   44m
harbor-trivy        ClusterIP   10.43.60.96     <none>        8080/TCP            44m
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
cat > harbor-https-values-k3s-mkcert.yaml <<'EOF'
# ============================================================
# Harbor Values - K3s(Traefik) + mkcert 本地证书
# 零修改可用（前提：域名是 harbor.k3s.local）
# https 访问 harbor.k3s.local
# ============================================================

expose:
  type: ingress
  tls:
    enabled: true
    certSource: secret
    secret:
      secretName: harbor-tls
  ingress:
    hosts:
      core: harbor.k3s.local
    ingressClassName: traefik
    annotations:
      # 监听 HTTPS
      traefik.ingress.kubernetes.io/router.entrypoints: websecure
      traefik.ingress.kubernetes.io/router.tls: "true"
      ingress.kubernetes.io/proxy-body-size: "0"
      nginx.ingress.kubernetes.io/proxy-body-size: "0"
      ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/ssl-redirect: "true"

externalURL: https://harbor.k3s.local

internalTLS:
  enabled: false

harborAdminPassword: "Harbor12345"

persistence:
  enabled: true
  resourcePolicy: "keep"
  persistentVolumeClaim:
    registry:
      storageClass: "local-path"
      accessMode: ReadWriteOnce
      size: 10Gi
    jobservice:
      jobLog:
        storageClass: "local-path"
        accessMode: ReadWriteOnce
        size: 1Gi
    database:
      storageClass: "local-path"
      accessMode: ReadWriteOnce
      size: 2Gi
    redis:
      storageClass: "local-path"
      accessMode: ReadWriteOnce
      size: 1Gi
    trivy:
      storageClass: "local-path"
      accessMode: ReadWriteOnce
      size: 5Gi

nginx:
  image:
    repository: docker.m.daocloud.io/goharbor/nginx-photon
    tag: v2.15.0
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 64Mi
      cpu: 50m
    limits:
      memory: 128Mi
      cpu: 100m

portal:
  image:
    repository: docker.m.daocloud.io/goharbor/harbor-portal
    tag: v2.15.0
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 64Mi
      cpu: 50m
    limits:
      memory: 128Mi
      cpu: 100m

core:
  image:
    repository: docker.m.daocloud.io/goharbor/harbor-core
    tag: v2.15.0
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 256Mi
      cpu: 100m
    limits:
      memory: 512Mi
      cpu: 500m

jobservice:
  image:
    repository: docker.m.daocloud.io/goharbor/harbor-jobservice
    tag: v2.15.0
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 128Mi
      cpu: 50m
    limits:
      memory: 256Mi
      cpu: 200m

registry:
  registry:
    image:
      repository: docker.m.daocloud.io/goharbor/registry-photon
      tag: v2.15.0
  controller:
    image:
      repository: docker.m.daocloud.io/goharbor/harbor-registryctl
      tag: v2.15.0
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 128Mi
      cpu: 50m
    limits:
      memory: 256Mi
      cpu: 200m

trivy:
  image:
    repository: docker.m.daocloud.io/goharbor/trivy-adapter-photon
    tag: v2.15.0
  nodeSelector:
    workload: apps
  resources:
    requests:
      memory: 128Mi
      cpu: 50m
    limits:
      memory: 256Mi
      cpu: 200m

database:
  internal:
    image:
      repository: docker.m.daocloud.io/goharbor/harbor-db
      tag: v2.15.0
    nodeSelector:
      workload: apps
    resources:
      requests:
        memory: 128Mi
        cpu: 100m
      limits:
        memory: 256Mi
        cpu: 300m

redis:
  internal:
    image:
      repository: docker.m.daocloud.io/goharbor/redis-photon
      tag: v2.15.0
    nodeSelector:
      workload: apps
    resources:
      requests:
        memory: 64Mi
        cpu: 50m
      limits:
        memory: 128Mi
        cpu: 100m

exporter:
  image:
    repository: docker.m.daocloud.io/goharbor/harbor-exporter
    tag: v2.15.0
  nodeSelector:
    workload: apps
EOF

3. 问题处理

3.1 helm介绍

helm 是一个基于k8s的包管理工具，它可以帮助用户快速地部署、升级和管理k8s应用。

3.2 问题1：Helm 拉取 prometheus-community 仓库超时解决

这是 GitHub Pages 国内访问慢的典型问题,prometheus-community 的 index 文件有 几十 MB,网络差就超时。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# 1. 换源
# 删除旧源
helm repo remove prometheus-community 2>/dev/null

# 加新源(Azure 中国)
helm repo add prometheus-community http://mirror.azure.cn/kubernetes/charts-prometheus-community/

# 更新
helm repo update

helm repo remove prometheus-community 2>/dev/null
helm repo add prometheus-community https://helm-charts.daocloud.io/prometheus-community
helm repo update
-----------------------------------------------------------------
# 2. 手动下载 index
# 找到要装的 chart(先用别的办法知道版本,或从 GitHub 看)
# https://github.com/prometheus-community/helm-charts/releases

# 下载 prometheus chart(版本号换成最新的)
VERSION=25.28.0
curl -LO https://gh-proxy.com/https://github.com/prometheus-community/helm-charts/releases/download/prometheus-${VERSION}/prometheus-${VERSION}.tgz

# 本地安装
helm install prometheus ./prometheus-${VERSION}.tgz \
  -f prometheus-values.yaml \
  -n monitoring \
  --create-namespace
-----------------------------------------------------------------

3.3 问题2：镜像拉取失败

DaoCloud/public-image-mirror
1
2
3
4
5
6
7
8
9
10
11
12
13
# 1. 配置 containerd 镜像加速
/etc/containerd/certs.d/registry.k8s.io/hosts.toml
server = "https://registry.k8s.io"

# 主 mirror: DaoCloud(路径透明,兼容性最好)
[host."https://k8s.m.daocloud.io"]
  capabilities = ["pull", "resolve"]
  # 不要 override_path

# 备用 mirror: 阿里云(某些镜像可能更快)
[host."https://registry.aliyuncs.com/google_containers"]
  capabilities = ["pull", "resolve"]
  override_path = true

3.4 问题3：Harbor 部署 关于内置(k3s)traefik 和 原生(k8s) ingress-nginx的区别

1. 内置(k3s)traefik 是 k3s 内置的 Ingress 控制器,它支持 k3s 的所有功能,包括负载均衡、服务发现等。
2. 原生(k8s) ingress-nginx 是 k8s 社区维护的 Ingress 控制器,它支持 k8s 的所有功能,包括负载均衡、服务发现等。
特性 系统 A (原生 K8s + Nginx) 系统 B (K3s + Traefik)
外部访问端口 必须是 30000+ (除非 HostNetwork) 标准的 80/443
负载均衡组件 依赖外部 (如硬件负载均衡)
Ingress 资源声明 ingressClassName: nginx ingressClassName: traefik
Harbor externalURL http://domain:30080 http://domain
性能消耗 较低 (iptables 转发) 略高 (多了一层 Klipper 代理)
特性 NodePort 方案 hostNetwork 方案
Service 类型 type: NodePort type: ClusterIP (或不暴露)
端口设置 nodePorts.http=30080 hostNetwork: true
DNS 策略 默认 dnsPolicy: ClusterFirstWithHostNet
访问地址 http://IP:30080 http://IP

3.5 问题4：Harbo支持http 访问

1. 创建一个单独的 HTTP→HTTPS 跳转 Ingress
2. HTTP 和 HTTPS 都直接访问（不跳转），需要两个独立的 Ingress，分别绑定不同 entrypoint
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# 1. 创建 HTTP→HTTPS 跳转 Ingress
cat > harbor-http-redirect.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: harbor-ingress-http-redirect
  namespace: harbor
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
    traefik.ingress.kubernetes.io/router.middlewares: harbor-redirect-https@kubernetescrd
spec:
  ingressClassName: traefik
  rules:
  - host: harbor.k3s.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: harbor-portal
            port:
              number: 80
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: redirect-https
  namespace: harbor
spec:
  redirectScheme:
    scheme: https
    permanent: true
EOF

kubectl apply -f harbor-http-redirect.yaml
# 访问效果
http://harbor.k3s.local  → 301 → https://harbor.k3s.local  ✅
https://harbor.k3s.local → 直接访问  ✅
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
浏览器/Docker
    │
    ├── https://harbor.k3s.local  → 节点:443
    │                                  │
    │                             Traefik websecure
    │                                  │
    │                             harbor-ingress（HTTPS）
    │                                  │
    │                             harbor-portal/core
    │
    └── http://harbor.k3s.local   → 节点:80
                                       │
                                  Traefik web
                                       │
                                  harbor-ingress-http（HTTP）
                                       │
                                  harbor-portal/core

# 2. 创建 HTTP Ingress
cat > harbor-http-ingress.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: harbor-ingress-http
  namespace: harbor
  annotations:
    # 只走 HTTP，不加 tls
    traefik.ingress.kubernetes.io/router.entrypoints: web
    ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
spec:
  ingressClassName: traefik
  rules:
  - host: harbor.k3s.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: harbor-portal
            port:
              number: 80
      - path: /api/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
      - path: /service/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
      - path: /v2/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
      - path: /c/
        pathType: Prefix
        backend:
          service:
            name: harbor-core
            port:
              number: 80
EOF

kubectl apply -f harbor-http-ingress.yaml

[xy@k3s-master01 Harbor]$ kubectl get ingress -n harbor
NAME                  CLASS     HOSTS              ADDRESS                                        PORTS     AGE
harbor-ingress        traefik   harbor.k3s.local   192.168.56.132,192.168.56.133,192.168.56.134   80, 443   98m
harbor-ingress-http   traefik   harbor.k3s.local   192.168.56.132,192.168.56.133,192.168.56.134   80        36s
k3s部署及应用
https://anyue967.github.io/posts/96937c05.html
作者
anyu967
发布于
2026年5月9日
许可协议