Istio
Think it through.

Istio is a service mesh — a dedicated infrastructure layer for handling service-to-service communication. It adds a sidecar Envoy proxy to each pod, intercepting all traffic. Features: traffic routing (canary, blue-green), security (mTLS, auth policies), observability (metrics, traces, logs), and resilience (retries, circuit breakers, fault injection). The control plane (istiod) manages proxy configuration. The data plane (Envoy proxies) handles all traffic.

A VirtualService defines routing rules for traffic to a destination. It specifies: (1) which hosts to match (service name, FQDN, wildcard), (2) route rules based on headers, URI, or weight, (3) destination subsets (via DestinationRule subsets), and (4) advanced features like retries, timeouts, and fault injection. Example: route 90% traffic to v1 and 10% to v2 for canary deployments. VirtualServices are the core traffic management primitive.

A DestinationRule defines policies that apply after routing: (1) Subsets — label-based groups of pods (e.g., v1, v2). (2) Traffic policy — connection pool settings, outlier detection (circuit breakers), load balancer algorithm (ROUND_ROBIN, LEAST_REQUEST, RANDOM, PASSTHROUGH). (3) TLS settings — mTLS mode (DISABLE, SIMPLE, MUTUAL, ISTIO_MUTUAL). Example: trafficPolicy: loadBalancer: simple: LEAST_REQUEST; connectionPool: tcp: maxConnections: 100. DestinationRules are always paired with VirtualServices.

Istio can enforce mTLS automatically between all sidecars. Modes: (1) DISABLE — plaintext. (2) ISTIO_MUTUAL — Istio certs (SPIFFE format), auto-rotated every 24h. (3) MUTUAL — custom CA certs. Enable globally: PeerAuthentication: mtls: mode: STRICT. Per-namespace: PeerAuthentication: metadata: namespace: prod. Per-port: portLevelMtls: 8080: mode: STRICT. The Citadel agent in istiod issues certificates via the Kubernetes CSR API. mTLS provides identity-based encryption between services — no app changes needed.

Istio automatically generates detailed metrics for all service-to-service traffic: request count, duration (latency), request size, response size, TCP byte counts, and gRPC streams. Metrics are enriched with labels: source/destination service, namespace, response code, protocol. Prometheus scrapes Envoy proxies via Istio's telemetry v2 (in-envoy stats). Key dashboards: (1) Istio Service Dashboard — per-service RPS, latency, errors. (2) Istio Workload Dashboard — per-workload metrics. (3) Istio Mesh Dashboard — aggregate mesh health. Istio also supports tracing (Jaeger/Zipkin) and access logging.

AuthorizationPolicy provides service-level access control (layer 7). Syntax: spec: selector: matchLabels: app: httpbin; rules: - from: - source: principals: ["cluster.local/ns/default/sa/sleep"]; to: - operation: methods: ["GET"] paths: ["/public/*"]. Actions: ALLOW, DENY, CUSTOM, AUDIT. Authorization policies are evaluated in order: CUSTOM → DENY → ALLOW. If no policy matches, the default action depends on the mesh config — typically ALLOW. For zero-trust: create a DENY-ALL policy first, then explicitly ALLOW needed traffic. Layer 7 awareness means you can restrict by JWT claims, IP blocks, namespaces, and service accounts.

Use VirtualService + DestinationRule: (1) DestinationRule defines subsets: subsets: - name: stable; labels: version: stable; - name: canary; labels: version: canary. (2) VirtualService routes traffic by weight: http: - route: - destination: host: myapp subset: stable weight: 90; - destination: host: myapp subset: canary weight: 10. (3) For HTTP header-based routing: match: - headers: version: exact: canary. (4) Gradual promotion: change weight from 10 → 25 → 50 → 75 → 100. No DNS changes needed. Combine with Flagger or Argo Rollouts for automated progressive delivery based on metrics and analysis.

The Istio ingress gateway is a dedicated Envoy proxy at the mesh edge — it runs as a separate Deployment (istio-ingressgateway), not as a sidecar. Unlike Kubernetes Ingress (which is a platform-agnostic API), the Istio Gateway allows: (1) Layer 4-7 routing with full Envoy capabilities, (2) TLS termination with mTLS passthrough, (3) VirtualService-based routing with match conditions (headers, URI, methods), (4) weight-based traffic splitting, and (5) integration with the Istio authorization/mTLS model. Define in Gateway CRD: spec: selector: istio: ingressgateway; servers: - port: number: 443; hosts: ["*.example.com"]; tls: mode: SIMPLE. The Gateway + VirtualService replaces the Ingress resource for mesh-aware ingress.

Strategy: (1) Canary upgrade — deploy new istiod version alongside old: istioctl install --set revision=canary. (2) Relabel namespaces to the new revision: istioctl proxy-status to verify proxies are connected to the correct istiod. (3) Gradually restart sidecars by rolling deployments: kubectl rollout restart deploy -n default. (4) Monitor with istioctl proxy-status and dashboard metrics during transition. (5) After all proxies are on the new revision, remove the old revision. For data plane upgrades: enable ambient mesh for gradual sidecar migration, or use istioctl upgrade (in-place). Key risks: CRD schema changes between versions must be validated first. Test with istioctl experimental precheck.

Istio: Envoy-based, more features (auth policies, fault injection, rich telemetry), but higher resource usage and complexity. Linkerd: Rust-based (linkerd2-proxy), lighter (~10MB memory per sidecar vs Istio's ~50MB), simpler to operate, supports TCP and HTTP/gRPC out of the box, but fewer features (no layer 7 auth policies, no fault injection, simpler traffic splitting). Istio is better for enterprise with complex security/traffic needs; Linkerd is better for teams wanting a simple, lightweight mesh with essential features. Both support mTLS, telemetry, and traffic splitting.