Active Nerds
Module #18 · expert Article

How Do You Build a Custom Admission Webhook from Scratch?: Architecture & Implementation

Build a TLS-secured HTTPS server that handles `AdmissionReview` JSON requests, register it with a `MutatingWebhookConfiguration` or `ValidatingWebhookConfigu...

20 min read+25 XPPublished 2026-05-10

Quick Answer:

Build a TLS-secured HTTPS server that handles AdmissionReview JSON requests, register it with a MutatingWebhookConfiguration or ValidatingWebhookConfiguration, and deploy it as a Pod inside the cluster with a cert-manager-managed certificate.


Detailed Answer:

A custom admission webhook is a standard HTTPS web server that receives AdmissionReview objects from the Kubernetes API server, processes them, and returns an AdmissionReview response with allowed: true/false (plus an optional JSON Patch for mutating webhooks). The four key requirements are: TLS (the API server only calls HTTPS endpoints), correct AdmissionReview API version handling, a caBundle in the webhook registration that matches your server's TLS certificate, and low latency (the API server has a configurable timeout, defaulting to 10 seconds, but you should target under 200ms). The easiest production path uses cert-manager to issue and rotate the webhook's TLS certificate automatically.


Deep Dive:

Full project structure:

webhook/
├── main.go
├── handler.go
├── cert/
│   └── tls.go
├── Dockerfile
├── deploy/
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── certificate.yaml        # cert-manager Certificate
│   └── webhookconfig.yaml

Step 1 — The webhook server (main.go):

package main

import (
    "context"
    "crypto/tls"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("/validate", validateHandler)
    mux.HandleFunc("/mutate",   mutateHandler)
    mux.HandleFunc("/healthz",  func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusOK)
    })

    cert, err := tls.LoadX509KeyPair(
        "/etc/webhook/certs/tls.crt",
        "/etc/webhook/certs/tls.key",
    )
    if err != nil {
        log.Fatalf("failed to load TLS keypair: %v", err)
    }

    server := &http.Server{
        Addr:    ":8443",
        Handler: mux,
        TLSConfig: &tls.Config{
            Certificates: []tls.Certificate{cert},
            MinVersion:   tls.VersionTLS13,
        },
        ReadTimeout:  5 * time.Second,
        WriteTimeout: 10 * time.Second,
        IdleTimeout:  60 * time.Second,
    }

    // Graceful shutdown
    go func() {
        log.Println("Webhook server listening on :8443")
        if err := server.ListenAndServeTLS("", ""); err != nil &&
            err != http.ErrServerClosed {
            log.Fatalf("server error: %v", err)
        }
    }()

    sig := make(chan os.Signal, 1)
    signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
    <-sig

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    server.Shutdown(ctx)
}

Step 2 — Validating handler (handler.go):

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "net/http"

    admissionv1 "k8s.io/api/admission/v1"
    appsv1      "k8s.io/api/apps/v1"
    metav1      "k8s.io/apimachinery/pkg/apis/meta/v1"
)

func validateHandler(w http.ResponseWriter, r *http.Request) {
    body, err := io.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "failed to read body", http.StatusBadRequest)
        return
    }

    var review admissionv1.AdmissionReview
    if err := json.Unmarshal(body, &review); err != nil {
        http.Error(w, "failed to decode AdmissionReview", http.StatusBadRequest)
        return
    }

    response := validateDeployment(review.Request)
    review.Response = response
    review.Response.UID = review.Request.UID

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(review)
}

func validateDeployment(req *admissionv1.AdmissionRequest) *admissionv1.AdmissionResponse {
    var deploy appsv1.Deployment
    if err := json.Unmarshal(req.Object.Raw, &deploy); err != nil {
        return deny(fmt.Sprintf("failed to parse Deployment: %v", err))
    }

    // Rule 1: Require resource limits on all containers
    for _, c := range deploy.Spec.Template.Spec.Containers {
        if c.Resources.Limits == nil {
            return deny(fmt.Sprintf(
                "container %q must define resource limits (CPU and memory)",
                c.Name,
            ))
        }
        if _, ok := c.Resources.Limits[corev1.ResourceCPU]; !ok {
            return deny(fmt.Sprintf("container %q missing CPU limit", c.Name))
        }
        if _, ok := c.Resources.Limits[corev1.ResourceMemory]; !ok {
            return deny(fmt.Sprintf("container %q missing memory limit", c.Name))
        }
    }

    // Rule 2: Require team label
    if _, ok := deploy.Labels["team"]; !ok {
        return deny("Deployment must have a 'team' label")
    }

    // Rule 3: Disallow latest tag
    for _, c := range deploy.Spec.Template.Spec.Containers {
        if strings.HasSuffix(c.Image, ":latest") || !strings.Contains(c.Image, ":") {
            return deny(fmt.Sprintf(
                "container %q uses 'latest' tag — pin to a specific digest or version",
                c.Name,
            ))
        }
    }

    return allow()
}

func allow() *admissionv1.AdmissionResponse {
    return &admissionv1.AdmissionResponse{Allowed: true}
}

func deny(reason string) *admissionv1.AdmissionResponse {
    return &admissionv1.AdmissionResponse{
        Allowed: false,
        Result: &metav1.Status{
            Code:    403,
            Message: reason,
        },
    }
}

Step 3 — cert-manager Certificate (deploy/certificate.yaml):

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: webhook-tls
  namespace: platform
spec:
  secretName: webhook-tls-secret
  duration: 8760h   # 1 year
  renewBefore: 720h # renew 30 days before expiry
  dnsNames:
    - webhook-svc.platform.svc
    - webhook-svc.platform.svc.cluster.local
  issuerRef:
    name: cluster-ca-issuer
    kind: ClusterIssuer

Step 4 — Deployment with cert volume mount:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: admission-webhook
  namespace: platform
spec:
  replicas: 2   # HA — never run a single replica webhook
  selector:
    matchLabels:
      app: admission-webhook
  template:
    metadata:
      labels:
        app: admission-webhook
    spec:
      # Webhook must stay up — use high-priority class
      priorityClassName: system-cluster-critical
      containers:
        - name: webhook
          image: mycompany/admission-webhook:v1.2.3
          ports:
            - containerPort: 8443
          volumeMounts:
            - name: tls-certs
              mountPath: /etc/webhook/certs
              readOnly: true
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8443
              scheme: HTTPS
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              cpu: 200m
              memory: 128Mi
      volumes:
        - name: tls-certs
          secret:
            secretName: webhook-tls-secret
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: admission-webhook

Step 5 — Webhook registration with caBundle injection:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: deployment-policy
  annotations:
    # cert-manager injects the caBundle automatically
    cert-manager.io/inject-ca-from: platform/webhook-tls
webhooks:
  - name: validate.deployments.mycompany.io
    admissionReviewVersions: ["v1"]
    clientConfig:
      service:
        name: webhook-svc
        namespace: platform
        path: /validate
        port: 443
    rules:
      - operations: ["CREATE", "UPDATE"]
        apiGroups: ["apps"]
        apiVersions: ["v1"]
        resources: ["deployments"]
    # Exempt the webhook itself to prevent deadlock
    namespaceSelector:
      matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: NotIn
          values: ["platform", "kube-system"]
    failurePolicy: Fail
    sideEffects: None
    timeoutSeconds: 5

Performance benchmarks to target:

| Metric | Target | Alert Threshold | |---|---|---| | p50 response latency | < 5ms | > 50ms | | p99 response latency | < 50ms | > 500ms | | Error rate | < 0.1% | > 1% | | Webhook pod CPU | < 100m | > 300m | | Replicas | ≥ 2 | < 2 |

Production lesson: Always add namespaceSelector to exclude kube-system and your webhook's own namespace. A webhook that intercepts its own Pod's creation and fails creates a deadlock — your webhook Pod can't start, so the webhook is down, so nothing can be admitted, including the fix.

Common mistake: Running a single replica webhook. If that Pod is rescheduled, all API requests that match your webhook rules are blocked (with failurePolicy: Fail) until the Pod is back. Always run 2+ replicas with topologySpreadConstraints.

Self-assessment: Can you implement this from scratch without referencing docs? Can you explain the caBundle requirement? Can you describe what happens when failurePolicy: Fail and your webhook is down?

Follow-up questions: Q4 (OPA/Gatekeeper), Q5 (Enforce image registry policy)


Finished reading?

Mark as complete to claim your +25 XP and track progress.