Proxies and routing
Traefik
Entrypoints, routers, middlewares and services, configured from providers, with the checks for a route that is not matching.
Cheatsheet #
| Task | Command or setting |
|---|---|
| What routes exist | curl -s localhost:8080/api/http/routers | jq -r '.[].rule' |
| Why is a router down | curl -s localhost:8080/api/http/routers | jq -r '.[] | select(.status!="enabled")' |
| Services and their servers | curl -s localhost:8080/api/http/services | jq |
| Is the provider loading | curl -s localhost:8080/api/overview | jq .providers |
| Enable the dashboard | --api.dashboard=true plus a secured router |
| Debug logs | --log.level=DEBUG |
| Access logs as JSON | --accesslog.format=json |
| Certificate status | jq '.letsencrypt.Certificates[].domain' acme.json |
| Route a container | label traefik.http.routers.app.rule=Host(\app.example.com`)` |
| Match a path prefix | rule=Host(`x`) && PathPrefix(`/api`) |
| Force a service port | traefik.http.services.app.loadbalancer.server.port=8080 |
| Strip a prefix | middleware stripprefix.prefixes=/api |
| Redirect to HTTPS | entrypoint http.redirections.entryPoint.to=websecure |
The pipeline #
A request arrives at an entrypoint (a port), matches a router (a rule), passes through middlewares (in the order listed), and reaches a service (a load-balanced set of servers). Configuration comes from providers: Docker labels, Kubernetes CRDs, files. Static configuration (entrypoints, providers, certificate resolvers) is read at start-up; dynamic configuration (routers, services, middlewares) is watched and applied live.
Everything that “does not work” is one of: the provider never produced the object, the rule does not match, a middleware rejected it, or the service has no healthy servers. The API answers all four.
curl -s localhost:8080/api/overview | jq
curl -s localhost:8080/api/http/routers | jq -r '.[] | [.name, .rule, .status] | @tsv'
curl -s localhost:8080/api/http/services | jq -r '.[] | [.name, (.loadBalancer.servers|length), .status] | @tsv'
curl -s localhost:8080/api/http/middlewares | jq -r '.[].name'Static configuration #
# traefik.yaml — read once at start-up
entryPoints:
web:
address: ":80"
http:
redirections:
entryPoint: { to: websecure, scheme: https, permanent: true }
websecure:
address: ":443"
http:
tls:
certResolver: letsencrypt
transport:
respondingTimeouts: { readTimeout: 30s, writeTimeout: 30s, idleTimeout: 180s }
providers:
kubernetesCRD:
allowCrossNamespace: false
docker:
exposedByDefault: false # opt in per container, never the reverse
file:
directory: /etc/traefik/dynamic
watch: true
certificatesResolvers:
letsencrypt:
acme:
email: ops@example.com
storage: /data/acme.json # chmod 600, on persistent storage
tlsChallenge: {}
api:
dashboard: true
metrics:
prometheus:
addEntryPointsLabels: true
addServicesLabels: true
accessLog:
format: json
filters: { statusCodes: ["400-599"], retryAttempts: true }
log:
level: INFOexposedByDefault: false is the single most important setting on the Docker provider: without it every container on the host becomes routable.
Routers and rules #
http:
routers:
api:
rule: "Host(`api.example.com`) && PathPrefix(`/v1`)"
entryPoints: [websecure]
middlewares: [ratelimit, secure-headers]
service: api
priority: 100
tls: { certResolver: letsencrypt }| Matcher | Example |
|---|---|
Host | Host(`api.example.com`) |
HostRegexp | HostRegexp(`^.+\.example\.com$`) |
PathPrefix | PathPrefix(`/api`) |
Path | Path(`/healthz`) |
Header | Header(`x-env`, `prod`) |
Query | Query(`debug`, `true`) |
ClientIP | ClientIP(`10.0.0.0/8`) |
Method | Method(`POST`) |
Longer rules win by default because priority defaults to the rule’s length. Set priority explicitly whenever two routers could match the same request — relying on length is how a catch-all silently swallows a specific route.
Services #
http:
services:
api:
loadBalancer:
servers:
- url: "http://10.0.0.5:8080"
- url: "http://10.0.0.6:8080"
healthCheck: { path: /healthz, interval: 10s, timeout: 3s }
sticky:
cookie: { name: srv, httpOnly: true, secure: true, sameSite: lax }
api-split:
weighted:
services:
- { name: api-v1, weight: 90 }
- { name: api-v2, weight: 10 }
api-mirror:
mirroring:
service: api-v1
mirrors: [{ name: api-v2, percent: 10 }]weighted is the canary mechanism; mirroring copies traffic and discards the response, so a new version can be load-tested with production traffic and no user impact.
Middlewares #
Middlewares run in the order listed on the router. Order decides behaviour: authentication before rate limiting protects the auth backend, rate limiting before authentication protects everything else.
http:
middlewares:
secure-headers:
headers:
stsSeconds: 31536000
frameDeny: true
contentTypeNosniff: true
referrerPolicy: no-referrer
ratelimit:
rateLimit: { average: 100, burst: 200, period: 1s }
basic-auth:
basicAuth: { usersFile: /etc/traefik/users } # htpasswd format
forward-auth:
forwardAuth:
address: http://auth:4180/verify
authResponseHeaders: [x-auth-user, x-auth-groups]
strip:
stripPrefix: { prefixes: ["/api"] }
retry:
retry: { attempts: 3, initialInterval: 100ms }
circuit:
circuitBreaker: { expression: "NetworkErrorRatio() > 0.30 || ResponseCodeRatio(500, 600, 0, 600) > 0.25" }
compress:
compress: {}
allowlist:
ipAllowList: { sourceRange: ["10.0.0.0/8", "192.168.0.0/16"] }Behind a load balancer, client IP matching needs the real address: set forwardedHeaders.trustedIPs on the entrypoint, or ipAllowList sees only the balancer.
Kubernetes #
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata: { name: api, namespace: apps }
spec:
entryPoints: [websecure]
routes:
- match: Host(`api.example.com`) && PathPrefix(`/v1`)
kind: Rule
priority: 100
middlewares:
- { name: secure-headers }
services:
- { name: api, port: 80 }
tls: { certResolver: letsencrypt }apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata: { name: secure-headers, namespace: apps }
spec:
headers: { stsSeconds: 31536000, frameDeny: true }Middlewares are namespaced; referencing one from another namespace needs namespace-name@kubernetescrd and allowCrossNamespace: true. Traefik also serves plain Ingress and, in current versions, Gateway API — see Gateway API if the goal is portability between controllers.
Docker labels #
services:
app:
image: myapp:1.0
labels:
- traefik.enable=true
- traefik.http.routers.app.rule=Host(`app.example.com`)
- traefik.http.routers.app.entrypoints=websecure
- traefik.http.routers.app.tls.certresolver=letsencrypt
- traefik.http.routers.app.middlewares=app-strip
- traefik.http.middlewares.app-strip.stripprefix.prefixes=/api
- traefik.http.services.app.loadbalancer.server.port=8080
networks: [edge]Traefik must share a network with the container, and loadbalancer.server.port is required whenever the image exposes more than one port.
Observability #
metrics:
prometheus:
buckets: [0.1, 0.3, 1.2, 5.0]
addEntryPointsLabels: true
addRoutersLabels: true
tracing:
otlp:
http: { endpoint: http://otel-collector:4318/v1/traces }Useful series: traefik_service_requests_total, traefik_service_request_duration_seconds_bucket, traefik_entrypoint_open_connections, traefik_service_server_up.
sum by (service) (rate(traefik_service_requests_total{code=~"5.."}[5m]))
/ sum by (service) (rate(traefik_service_requests_total[5m]))Production notes #
Run at least two replicas behind a load balancer; ACME storage in acme.json is not shared safely between them, so use a DNS-01 resolver with distributed storage or terminate certificates elsewhere (cert-manager) when scaling out.
Never expose the API or dashboard on a public entrypoint without authentication: --api.insecure=true binds an unauthenticated dashboard on port 8080.
Oneliners #
# Routers that are not enabled, with their error
curl -s localhost:8080/api/http/routers | jq -r '.[] | select(.status!="enabled") | [.name, (.error//[]|join("; "))] | @tsv'
# Services with zero servers
curl -s localhost:8080/api/http/services | jq -r '.[] | select((.loadBalancer.servers//[])|length==0) | .name'
# Which router would match a host
curl -s localhost:8080/api/http/routers | jq -r '.[] | select(.rule | test("api.example.com")) | [.name, .priority, .rule] | @tsv'
# Certificate domains and expiry from acme.json
jq -r '.letsencrypt.Certificates[] | .domain.main' /data/acme.json
# Tail access logs for 5xx only
tail -f /var/log/traefik/access.log | jq -r 'select(.DownstreamStatus >= 500) | [.time, .RouterName, .DownstreamStatus, .RequestPath] | @tsv'
# Slowest routes in the last log file
jq -r '[.Duration/1000000, .RouterName, .RequestPath] | @tsv' access.log | sort -rn | head
# Requests per router
jq -r '.RouterName' access.log | sort | uniq -c | sort -rn | head
# Confirm a container is visible to the Docker provider
docker inspect app | jq -r '.[0].Config.Labels | with_entries(select(.key|startswith("traefik")))'
# Reload check: file provider syntax
traefik --configFile=/etc/traefik/traefik.yaml --log.level=DEBUG 2>&1 | head -20
# Test a route without DNS
curl -H 'Host: api.example.com' -k https://127.0.0.1/v1/healthz