KubeSphere Gateway
Overview
Provides external access management (ingress) for KubeSphere using ingress-nginx. Supports three-tier gateway management:
| Tier | Scope | Name Pattern | Namespace | Label |
|---|---|---|---|
| Cluster | Entire cluster | kubesphere-router-cluster | kubesphere-controls-system | kubesphere.io/gateway-type=cluster |
| Workspace | Single workspace | kubesphere-router-workspace-{workspace} | kubesphere-controls-system | kubesphere.io/gateway-type=workspace |
| Project | Single project/namespace | kubesphere-router-{namespace} | kubesphere-controls-system | kubesphere.io/gateway-type=project |
Each gateway is a standalone Helm release of ingress-nginx. The gateway-controller-manager manages the lifecycle (install/upgrade/uninstall) via Helm.
Core CRDs
Monitoring Integration
Gateway exposes NGINX metrics (requests, 4xx/5xx, latency P50/P90/P99) via Prometheus. Requires the whizard-monitoring extension (optional dependency).
Before You Start
Check if Gateway extension is already installed:
kubectl get installplans.kubesphere.io gateway --ignore-not-found
If found, upgrading is supported — just select a newer version in Step 1.
Installation
Step 1: Detect and Select Version
ALL_VERSIONS=$(kubectl get extensionversions.kubesphere.io \
-l kubesphere.io/extension-ref=gateway \
-o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V)
LATEST_STABLE=$(echo "$ALL_VERSIONS" | grep -v -E 'alpha|beta|rc' | tail -1)
if [ -z "$LATEST_STABLE" ]; then
LATEST_STABLE=$(echo "$ALL_VERSIONS" | tail -1)
fi
echo "Available versions:"
echo "$ALL_VERSIONS"
echo ""
echo "Latest stable: $LATEST_STABLE"
This sets ALL_VERSIONS and LATEST_STABLE. Use SELECTED_VERSION for the version chosen.
Use the question tool:
$LATEST_STABLE (Recommended) — accept the auto-detected version- (custom) — type a specific version; validate it against the printed list
Step 2: Detect and Select Clusters
CLUSTER_DATA=$(kubectl get clusters.cluster.kubesphere.io \
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}')
READY_CLUSTERS=$(echo "$CLUSTER_DATA" | awk -F'\t' '$2 == "True" {print $1}')
CLUSTER_COUNT=$(echo "$READY_CLUSTERS" | wc -l)
HOST_CLUSTER=$(kubectl get clusters.cluster.kubesphere.io \
-l 'cluster-role.kubesphere.io/host' \
-o jsonpath='{.items[0].metadata.name}' || echo "")
echo "Ready clusters:"
echo "$READY_CLUSTERS"
echo ""
echo "Cluster count: $CLUSTER_COUNT"
echo "Host cluster: $HOST_CLUSTER"
This sets READY_CLUSTERS, CLUSTER_COUNT, HOST_CLUSTER.
- 1 cluster → skip selection, auto-use it. Set
TARGET_CLUSTERS="$HOST_CLUSTER"
- Multiple clusters → use
question with multiple: true:
- All clusters →
TARGET_CLUSTERS="$READY_CLUSTERS"
- Host cluster only →
TARGET_CLUSTERS="$HOST_CLUSTER"
- (custom) — validate each name against
$READY_CLUSTERS
Step 3: Generate and Apply InstallPlan
./scripts/generate-installplan.sh "$SELECTED_VERSION" "$TARGET_CLUSTERS"
This generates the YAML to /tmp/gateway-installplan.yaml, runs --dry-run=server, then prints the apply command.
For configurable extension values (ingress-nginx default settings, image registry, upgrade tool config, etc.), see references/extension-values.md.
Apply it:
kubectl apply -f /tmp/gateway-installplan.yaml
Tell the user "Installing". Then ask if they want to check status. If yes:
./scripts/check-status.sh poll
Status Checking
| Purpose | Command |
|---|
| Single snapshot | ./scripts/check-status.sh quick |
| Wait until complete (5min timeout) | ./scripts/check-status.sh poll |
Logic:
- All
Installed → ✓ success
- Any
Failed → ✗ prints full status
- Timeout (300s) → ⚠ prints current status
- In progress → prints every 10s
Uninstallation
⚠ Always confirm with the user before proceeding.
Uninstall from all clusters
if ! kubectl get installplans.kubesphere.io gateway &>/dev/null; then
echo "Gateway is not installed."
exit 0
fi
Confirm with the user, then delete:
kubectl delete installplans.kubesphere.io gateway --ignore-not-found
Verify cleanup:
./scripts/verify-uninstall.sh
Success criteria:
- InstallPlan is deleted
- No active pods remain in
extension-gateway namespace
Uninstall from specific clusters
WARNING: Do NOT delete the InstallPlan. Only remove target clusters from the placement list.
Confirm which clusters to remove, compute remaining clusters, then patch:
kubectl patch installplans.kubesphere.io gateway --type='json' \
-p='[{"op": "replace", "path": "/spec/clusterScheduling/placement/clusters", "value": ["<REMAINING_CLUSTER_1>", "<REMAINING_CLUSTER_2>"]}]'
Success: patch returns OK + removed clusters no longer in .status.clusterSchedulingStatuses.
Gateway Operations
List Gateways
Gateways are organized by tier (see Overview), with each tier identified by the label kubesphere.io/gateway-type. List them grouped by tier:
echo "=== Cluster Gateway ==="
kubectl get gateways.gateway.kubesphere.io -n kubesphere-controls-system \
-l kubesphere.io/gateway-type=cluster
echo -e "\n=== Workspace Gateways ==="
kubectl get gateways.gateway.kubesphere.io -n kubesphere-controls-system \
-l kubesphere.io/gateway-type=workspace
echo -e "\n=== Project Gateways ==="
kubectl get gateways.gateway.kubesphere.io -n kubesphere-controls-system \
-l kubesphere.io/gateway-type=project
Check Gateway Status
Pick a gateway name from the List Gateways output and run:
GW_NS="kubesphere-controls-system"
GW_NAME="<gateway-name-from-list>"
# app.kubernetes.io/instance uses the Helm release name if available, otherwise the Gateway name
GW_INSTANCE=$(kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o jsonpath='{.status.helmRelease.name}' 2>/dev/null)
if [ -z "$GW_INSTANCE" ]; then
GW_INSTANCE="$GW_NAME"
fi
kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o wide
kubectl describe gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME
kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o yaml
kubectl get pods -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE"
Gateway states:
| State | Meaning |
|---|
Creating | First-time Helm install in progress |
Updating | Helm upgrade in progress (spec changed) |
Running | Fully operational (all replicas available) |
Faulted | Deployment missing, stopped unexpectedly, or health probe timeout |
Stopped | Scaled to zero replicas intentionally |
Troubleshooting
Set $GW_NAME according to the gateway tier being troubleshot (see naming rules in Overview):
- Cluster →
GW_NAME=kubesphere-router-cluster
- Workspace →
GW_NAME=kubesphere-router-workspace-${WORKSPACE}
- Project →
GW_NAME=kubesphere-router-${NAMESPACE}
Common namespace: GW_NS=kubesphere-controls-system
$GW_INSTANCE is auto-resolved from status.helmRelease.name (falls back to $GW_NAME). If not yet set, run:
GW_INSTANCE=$(kubectl get gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME -o jsonpath='{.status.helmRelease.name}' 2>/dev/null)
if [ -z "$GW_INSTANCE" ]; then
GW_INSTANCE="$GW_NAME"
fi
Gateway stuck in Creating or Updating state
kubectl describe gateways.gateway.kubesphere.io -n $GW_NS $GW_NAME
kubectl logs -n extension-gateway -l app=gateway-controller-manager --tail=200 | grep -iE "(error|helm|install|upgrade|reconcile)"
kubectl get deployment -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE,app.kubernetes.io/component=controller"
kubectl get configmap -n $GW_NS $GW_NAME -o yaml
Common causes: Chart ConfigMap missing/corrupted, Helm wrapper timeout, invalid spec.values.
Gateway shows Faulted state
kubectl get deployment -n $GW_NS $GW_NAME -o wide
kubectl describe deployment -n $GW_NS $GW_NAME
kubectl get pods -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -o wide
POD_NAME=$(kubectl get pods -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -o jsonpath='{.items[0].metadata.name}')
kubectl describe pod -n $GW_NS $POD_NAME
kubectl logs -n $GW_NS $POD_NAME --tail=100
Common causes: Image pull failure, resource constraints, port conflicts, missing ConfigMap/Secret.
Gateway pod crash-looping / CrashLoopBackOff
kubectl logs -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" --tail=100 --previous
kubectl get events -n $GW_NS --sort-by='.lastTimestamp' | tail -20
kubectl exec -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -- cat /etc/nginx/nginx.conf 2>/dev/null | head -50
kubectl get configmap -n $GW_NS -l "app.kubernetes.io/instance=$GW_INSTANCE" -o yaml
Common causes: Misconfigured nginx config, port conflicts, resource limits (OOMKilled), missing dependencies (ConfigMap/Secret).
Log search not working
Gateway log search proxies to whizard-telemetry-apiserver:
kubectl get configmap -n extension-gateway gateway-agent-backend-config -o yaml
kubectl get pods -n extension-whizard-telemetry
kubectl get svc -n extension-whizard-telemetry whizard-telemetry-apiserver
kubectl logs -n extension-gateway -l app=gateway-apiserver --tail=100 | grep -iE "(log|search|whizard|proxy)"
Common causes: Whizard-telemetry not installed or not running, misconfigured gateway-agent-backend-config, network policy blocking cross-namespace traffic.
Gateway controller not reconciling
kubectl get pods -n extension-gateway -l app=gateway-controller-manager
kubectl logs -n extension-gateway -l app=gateway-controller-manager --tail=200
kubectl get validatingwebhookconfiguration -l "app.kubernetes.io/managed-by=Helm,kubesphere.io/extension-ref=gateway"
kubectl get deployment -n extension-gateway -l app=gateway-controller-manager -o yaml
Common causes: Controller pod not running, webhook configuration blocking updates, Helm release state mismatch, RBAC permission issues.