How to Set Up mTLS for Microservices

Regular TLS authenticates the server to the client (the client verifies the server’s certificate). Mutual TLS (mTLS) adds client authentication — the server also verifies the client’s certificate. For microservices, this means only services with a valid certificate from your private CA can call each other, even if an attacker reaches your internal network.

How mTLS Works

Client Service                    Server Service
     │                                   │
     │── Client certificate ──────────→  │ Verifies client cert against CA
     │← Server certificate ──────────── │
     │ Verifies server cert against CA   │
     │                                   │
     │═══════════ Encrypted ════════════ │ Both sides authenticated

Both the caller and the callee present certificates signed by the same private CA. If a service does not have a valid certificate, the TLS handshake fails — before any application data is exchanged.

Step 1: Set Up a Private Certificate Authority

# Create CA private key and certificate
mkdir -p /etc/mtls-ca/{certs,private}
chmod 700 /etc/mtls-ca/private

# CA private key
openssl genrsa -out /etc/mtls-ca/private/ca.key 4096
chmod 400 /etc/mtls-ca/private/ca.key

# CA certificate (valid 10 years)
openssl req -x509 -new -nodes \
  -key /etc/mtls-ca/private/ca.key \
  -sha256 -days 3650 \
  -out /etc/mtls-ca/certs/ca.crt \
  -subj "/C=US/O=My Company/CN=Internal Services CA"

# Verify
openssl x509 -in /etc/mtls-ca/certs/ca.crt -text -noout | head -20

Step 2: Issue Service Certificates

#!/bin/bash
# issue-cert.sh <service-name>
# Issues a certificate for a service, valid 90 days

SERVICE="$1"
CA_KEY="/etc/mtls-ca/private/ca.key"
CA_CERT="/etc/mtls-ca/certs/ca.crt"
OUT_DIR="/etc/mtls-certs/${SERVICE}"

mkdir -p "$OUT_DIR"

# Service private key
openssl genrsa -out "${OUT_DIR}/${SERVICE}.key" 2048

# CSR
openssl req -new \
  -key "${OUT_DIR}/${SERVICE}.key" \
  -out "${OUT_DIR}/${SERVICE}.csr" \
  -subj "/C=US/O=My Company/CN=${SERVICE}"

# Certificate config (include SANs for Kubernetes service discovery)
cat > "${OUT_DIR}/${SERVICE}.ext" << EOF
subjectAltName = DNS:${SERVICE}, DNS:${SERVICE}.default.svc.cluster.local, DNS:localhost
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
EOF

# Sign with CA
openssl x509 -req \
  -in "${OUT_DIR}/${SERVICE}.csr" \
  -CA "$CA_CERT" \
  -CAkey "$CA_KEY" \
  -CAcreateserial \
  -out "${OUT_DIR}/${SERVICE}.crt" \
  -days 90 \
  -sha256 \
  -extfile "${OUT_DIR}/${SERVICE}.ext"

echo "Issued: ${OUT_DIR}/${SERVICE}.crt"
openssl x509 -in "${OUT_DIR}/${SERVICE}.crt" -noout -dates

chmod +x issue-cert.sh
./issue-cert.sh user-service
./issue-cert.sh order-service
./issue-cert.sh payment-service

Step 3: Configure nginx for mTLS

# nginx.conf — server that requires client certificates
server {
    listen 443 ssl;
    server_name user-service.internal;

    # Server certificate
    ssl_certificate     /etc/mtls-certs/user-service/user-service.crt;
    ssl_certificate_key /etc/mtls-certs/user-service/user-service.key;

    # CA that signs valid client certificates
    ssl_client_certificate /etc/mtls-ca/certs/ca.crt;

    # Require client certificate — connections without a valid cert are rejected
    ssl_verify_client on;
    ssl_verify_depth 2;

    # TLS hardening
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
    ssl_prefer_server_ciphers off;

    location / {
        # Pass the client's CN in a header for application-level logging
        proxy_set_header X-Client-CN $ssl_client_s_dn_cn;
        proxy_pass http://user-service-backend;
    }
}

Step 4: Configure a Client Service (curl / Python)

# Test mTLS with curl
curl --cert /etc/mtls-certs/order-service/order-service.crt \
     --key /etc/mtls-certs/order-service/order-service.key \
     --cacert /etc/mtls-ca/certs/ca.crt \
     https://user-service.internal/api/users

# Without a valid cert — connection is rejected:
curl --cacert /etc/mtls-ca/certs/ca.crt https://user-service.internal/api/users
# curl: (56) OpenSSL SSL_read: error:14094412:SSL routines: SSL3_READ_BYTES:sslv3 alert bad certificate

# Python requests with mTLS
import requests

response = requests.get(
    "https://user-service.internal/api/users",
    cert=(
        "/etc/mtls-certs/order-service/order-service.crt",
        "/etc/mtls-certs/order-service/order-service.key"
    ),
    verify="/etc/mtls-ca/certs/ca.crt"
)
print(response.json())

// Go HTTP client with mTLS
package main

import (
    "crypto/tls"
    "crypto/x509"
    "net/http"
    "os"
)

func mtlsClient() (*http.Client, error) {
    cert, err := tls.LoadX509KeyPair(
        "/etc/mtls-certs/order-service/order-service.crt",
        "/etc/mtls-certs/order-service/order-service.key",
    )
    if err != nil {
        return nil, err
    }

    caCert, _ := os.ReadFile("/etc/mtls-ca/certs/ca.crt")
    caCertPool := x509.NewCertPool()
    caCertPool.AppendCertsFromPEM(caCert)

    tlsConfig := &tls.Config{
        Certificates: []tls.Certificate{cert},
        RootCAs:      caCertPool,
        MinVersion:   tls.VersionTLS12,
    }

    return &http.Client{
        Transport: &http.Transport{TLSClientConfig: tlsConfig},
    }, nil
}

Step 5: Automate Certificate Rotation with cert-manager (Kubernetes)

In Kubernetes, cert-manager handles certificate issuance and rotation automatically.

# issuer.yaml — ClusterIssuer using your private CA
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: internal-ca
spec:
  ca:
    secretName: internal-ca-key-pair
---
# Create the secret with your CA
# kubectl create secret tls internal-ca-key-pair \
#   --cert=/etc/mtls-ca/certs/ca.crt \
#   --key=/etc/mtls-ca/private/ca.key \
#   -n cert-manager

# certificate.yaml — request a cert for user-service
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: user-service-cert
  namespace: production
spec:
  secretName: user-service-tls
  duration: 2160h       # 90 days
  renewBefore: 360h     # renew 15 days before expiry
  subject:
    organizations: ["My Company"]
  dnsNames:
    - user-service
    - user-service.production.svc.cluster.local
  usages:
    - digital signature
    - key encipherment
    - server auth
    - client auth
  issuerRef:
    name: internal-ca
    kind: ClusterIssuer

kubectl apply -f issuer.yaml certificate.yaml

# cert-manager creates the secret automatically
kubectl get secret user-service-tls -n production -o yaml

Verifying mTLS in Production

# Check that a service certificate is valid and signed by your CA
openssl verify -CAfile /etc/mtls-ca/certs/ca.crt /etc/mtls-certs/user-service/user-service.crt
# user-service.crt: OK

# Check certificate expiry for all services
for service in user-service order-service payment-service; do
  expiry=$(openssl x509 -in /etc/mtls-certs/${service}/${service}.crt -noout -enddate | cut -d= -f2)
  echo "${service}: expires ${expiry}"
done

# Test mTLS from within a pod
kubectl run mtls-test --image=curlimages/curl --rm -it --restart=Never -- \
  curl --cert /path/to/client.crt --key /path/to/client.key \
  --cacert /path/to/ca.crt https://user-service/health

Istio Service Mesh mTLS (Zero-Config Option)

Running cert-manager and manually configuring nginx mTLS for every service is manageable at 5 services, painful at 50. Istio automates mTLS across your entire mesh with a single policy — every pod gets a sidecar that handles certificates without any application code changes.

# Install Istio with strict mTLS mode
istioctl install --set profile=default -y

# Enable strict mTLS for the production namespace
kubectl apply -f - <<'EOF'
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT
EOF

In STRICT mode, any pod without a valid Istio-issued certificate is rejected at the network level before the request reaches your application. To verify:

# Check the mTLS mode for a service
istioctl x describe service user-service.production

# Inspect the certificate Istio issued to a pod
istioctl proxy-config secret deploy/user-service -n production

# Test from inside the mesh (should succeed)
kubectl exec -it debug-pod -n production -- \
  curl -s http://user-service/health

# Test from outside the mesh (should fail with 503 or connection reset)
kubectl exec -it debug-pod -n default -- \
  curl -s http://user-service.production/health

For mixed environments where you are migrating services gradually, set mode: PERMISSIVE first (accepts both mTLS and plaintext), then switch to STRICT once all sidecars are deployed.

Certificate Renewal Automation Without Downtime

Short-lived certificates (90 days or less) reduce the blast radius of a key compromise, but manual renewal causes outages. The correct pattern uses overlapping validity windows and pre-rotation.

With cert-manager’s renewBefore field set to 15 days, the certificate rotates well before expiry. Your nginx or app server must reload the new certificate file without dropping connections:

# nginx: reload config without connection drops
sudo nginx -t && sudo nginx -s reload

# For Go services — watch for file changes and reload the tls.Certificate
# Use fsnotify to trigger a reload of the TLS config in the HTTP server

For services that embed certificates in memory at startup, add a SIGHUP handler:

// main.go — reload TLS cert on SIGHUP
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGHUP)
go func() {
    for range sigChan {
        newCert, err := tls.LoadX509KeyPair(certFile, keyFile)
        if err != nil {
            log.Printf("cert reload error: %v", err)
            continue
        }
        certMu.Lock()
        currentCert = &newCert
        certMu.Unlock()
        log.Println("TLS certificate reloaded")
    }
}()

Monitor expiry across all services proactively:

#!/bin/bash
# check-cert-expiry.sh — alert on certs expiring within 14 days
for service in user-service order-service payment-service; do
    expiry=$(openssl x509 \
        -in /etc/mtls-certs/${service}/${service}.crt \
        -noout -enddate 2>/dev/null | cut -d= -f2)
    expiry_epoch=$(date -d "$expiry" +%s 2>/dev/null || date -j -f "%b %d %H:%M:%S %Y %Z" "$expiry" +%s)
    now_epoch=$(date +%s)
    days_left=$(( (expiry_epoch - now_epoch) / 86400 ))
    if [[ $days_left -lt 14 ]]; then
        echo "WARNING: ${service} cert expires in ${days_left} days (${expiry})"
    else
        echo "OK: ${service} cert expires in ${days_left} days"
    fi
done