Software Engineering Wiki

Proxies and routing

Traefik

Entrypoints, routers, middlewares and services, configured from providers, with the checks for a route that is not matching.

Cheatsheet #

TaskCommand or setting
What routes existcurl -s localhost:8080/api/http/routers | jq -r '.[].rule'
Why is a router downcurl -s localhost:8080/api/http/routers | jq -r '.[] | select(.status!="enabled")'
Services and their serverscurl -s localhost:8080/api/http/services | jq
Is the provider loadingcurl -s localhost:8080/api/overview | jq .providers
Enable the dashboard--api.dashboard=true plus a secured router
Debug logs--log.level=DEBUG
Access logs as JSON--accesslog.format=json
Certificate statusjq '.letsencrypt.Certificates[].domain' acme.json
Route a containerlabel traefik.http.routers.app.rule=Host(\app.example.com`)`
Match a path prefixrule=Host(`x`) && PathPrefix(`/api`)
Force a service porttraefik.http.services.app.loadbalancer.server.port=8080
Strip a prefixmiddleware stripprefix.prefixes=/api
Redirect to HTTPSentrypoint http.redirections.entryPoint.to=websecure

The pipeline #

A request arrives at an entrypoint (a port), matches a router (a rule), passes through middlewares (in the order listed), and reaches a service (a load-balanced set of servers). Configuration comes from providers: Docker labels, Kubernetes CRDs, files. Static configuration (entrypoints, providers, certificate resolvers) is read at start-up; dynamic configuration (routers, services, middlewares) is watched and applied live.

Everything that “does not work” is one of: the provider never produced the object, the rule does not match, a middleware rejected it, or the service has no healthy servers. The API answers all four.

curl -s localhost:8080/api/overview | jq
curl -s localhost:8080/api/http/routers  | jq -r '.[] | [.name, .rule, .status] | @tsv'
curl -s localhost:8080/api/http/services | jq -r '.[] | [.name, (.loadBalancer.servers|length), .status] | @tsv'
curl -s localhost:8080/api/http/middlewares | jq -r '.[].name'

Static configuration #

# traefik.yaml — read once at start-up
entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint: { to: websecure, scheme: https, permanent: true }
  websecure:
    address: ":443"
    http:
      tls:
        certResolver: letsencrypt
    transport:
      respondingTimeouts: { readTimeout: 30s, writeTimeout: 30s, idleTimeout: 180s }

providers:
  kubernetesCRD:
    allowCrossNamespace: false
  docker:
    exposedByDefault: false          # opt in per container, never the reverse
  file:
    directory: /etc/traefik/dynamic
    watch: true

certificatesResolvers:
  letsencrypt:
    acme:
      email: ops@example.com
      storage: /data/acme.json       # chmod 600, on persistent storage
      tlsChallenge: {}

api:
  dashboard: true
metrics:
  prometheus:
    addEntryPointsLabels: true
    addServicesLabels: true
accessLog:
  format: json
  filters: { statusCodes: ["400-599"], retryAttempts: true }
log:
  level: INFO

exposedByDefault: false is the single most important setting on the Docker provider: without it every container on the host becomes routable.

Routers and rules #

http:
  routers:
    api:
      rule: "Host(`api.example.com`) && PathPrefix(`/v1`)"
      entryPoints: [websecure]
      middlewares: [ratelimit, secure-headers]
      service: api
      priority: 100
      tls: { certResolver: letsencrypt }
MatcherExample
HostHost(`api.example.com`)
HostRegexpHostRegexp(`^.+\.example\.com$`)
PathPrefixPathPrefix(`/api`)
PathPath(`/healthz`)
HeaderHeader(`x-env`, `prod`)
QueryQuery(`debug`, `true`)
ClientIPClientIP(`10.0.0.0/8`)
MethodMethod(`POST`)

Longer rules win by default because priority defaults to the rule’s length. Set priority explicitly whenever two routers could match the same request — relying on length is how a catch-all silently swallows a specific route.

Services #

http:
  services:
    api:
      loadBalancer:
        servers:
          - url: "http://10.0.0.5:8080"
          - url: "http://10.0.0.6:8080"
        healthCheck: { path: /healthz, interval: 10s, timeout: 3s }
        sticky:
          cookie: { name: srv, httpOnly: true, secure: true, sameSite: lax }
    api-split:
      weighted:
        services:
          - { name: api-v1, weight: 90 }
          - { name: api-v2, weight: 10 }
    api-mirror:
      mirroring:
        service: api-v1
        mirrors: [{ name: api-v2, percent: 10 }]

weighted is the canary mechanism; mirroring copies traffic and discards the response, so a new version can be load-tested with production traffic and no user impact.

Middlewares #

Middlewares run in the order listed on the router. Order decides behaviour: authentication before rate limiting protects the auth backend, rate limiting before authentication protects everything else.

http:
  middlewares:
    secure-headers:
      headers:
        stsSeconds: 31536000
        frameDeny: true
        contentTypeNosniff: true
        referrerPolicy: no-referrer
    ratelimit:
      rateLimit: { average: 100, burst: 200, period: 1s }
    basic-auth:
      basicAuth: { usersFile: /etc/traefik/users }        # htpasswd format
    forward-auth:
      forwardAuth:
        address: http://auth:4180/verify
        authResponseHeaders: [x-auth-user, x-auth-groups]
    strip:
      stripPrefix: { prefixes: ["/api"] }
    retry:
      retry: { attempts: 3, initialInterval: 100ms }
    circuit:
      circuitBreaker: { expression: "NetworkErrorRatio() > 0.30 || ResponseCodeRatio(500, 600, 0, 600) > 0.25" }
    compress:
      compress: {}
    allowlist:
      ipAllowList: { sourceRange: ["10.0.0.0/8", "192.168.0.0/16"] }

Behind a load balancer, client IP matching needs the real address: set forwardedHeaders.trustedIPs on the entrypoint, or ipAllowList sees only the balancer.

Kubernetes #

apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata: { name: api, namespace: apps }
spec:
  entryPoints: [websecure]
  routes:
    - match: Host(`api.example.com`) && PathPrefix(`/v1`)
      kind: Rule
      priority: 100
      middlewares:
        - { name: secure-headers }
      services:
        - { name: api, port: 80 }
  tls: { certResolver: letsencrypt }
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata: { name: secure-headers, namespace: apps }
spec:
  headers: { stsSeconds: 31536000, frameDeny: true }

Middlewares are namespaced; referencing one from another namespace needs namespace-name@kubernetescrd and allowCrossNamespace: true. Traefik also serves plain Ingress and, in current versions, Gateway API — see Gateway API if the goal is portability between controllers.

Docker labels #

services:
  app:
    image: myapp:1.0
    labels:
      - traefik.enable=true
      - traefik.http.routers.app.rule=Host(`app.example.com`)
      - traefik.http.routers.app.entrypoints=websecure
      - traefik.http.routers.app.tls.certresolver=letsencrypt
      - traefik.http.routers.app.middlewares=app-strip
      - traefik.http.middlewares.app-strip.stripprefix.prefixes=/api
      - traefik.http.services.app.loadbalancer.server.port=8080
    networks: [edge]

Traefik must share a network with the container, and loadbalancer.server.port is required whenever the image exposes more than one port.

Observability #

metrics:
  prometheus:
    buckets: [0.1, 0.3, 1.2, 5.0]
    addEntryPointsLabels: true
    addRoutersLabels: true
tracing:
  otlp:
    http: { endpoint: http://otel-collector:4318/v1/traces }

Useful series: traefik_service_requests_total, traefik_service_request_duration_seconds_bucket, traefik_entrypoint_open_connections, traefik_service_server_up.

sum by (service) (rate(traefik_service_requests_total{code=~"5.."}[5m]))
  / sum by (service) (rate(traefik_service_requests_total[5m]))

Production notes #

Run at least two replicas behind a load balancer; ACME storage in acme.json is not shared safely between them, so use a DNS-01 resolver with distributed storage or terminate certificates elsewhere (cert-manager) when scaling out.

Never expose the API or dashboard on a public entrypoint without authentication: --api.insecure=true binds an unauthenticated dashboard on port 8080.

Oneliners #

# Routers that are not enabled, with their error
curl -s localhost:8080/api/http/routers | jq -r '.[] | select(.status!="enabled") | [.name, (.error//[]|join("; "))] | @tsv'

# Services with zero servers
curl -s localhost:8080/api/http/services | jq -r '.[] | select((.loadBalancer.servers//[])|length==0) | .name'

# Which router would match a host
curl -s localhost:8080/api/http/routers | jq -r '.[] | select(.rule | test("api.example.com")) | [.name, .priority, .rule] | @tsv'

# Certificate domains and expiry from acme.json
jq -r '.letsencrypt.Certificates[] | .domain.main' /data/acme.json

# Tail access logs for 5xx only
tail -f /var/log/traefik/access.log | jq -r 'select(.DownstreamStatus >= 500) | [.time, .RouterName, .DownstreamStatus, .RequestPath] | @tsv'

# Slowest routes in the last log file
jq -r '[.Duration/1000000, .RouterName, .RequestPath] | @tsv' access.log | sort -rn | head

# Requests per router
jq -r '.RouterName' access.log | sort | uniq -c | sort -rn | head

# Confirm a container is visible to the Docker provider
docker inspect app | jq -r '.[0].Config.Labels | with_entries(select(.key|startswith("traefik")))'

# Reload check: file provider syntax
traefik --configFile=/etc/traefik/traefik.yaml --log.level=DEBUG 2>&1 | head -20

# Test a route without DNS
curl -H 'Host: api.example.com' -k https://127.0.0.1/v1/healthz

Last updated 15 September 2026 · Edit this page