Akuity Agent Fails to Start with x509: certificate signed by unknown authority

Symptoms

akuity-agent pods fail to start or repeatedly crash. Cluster remains disconnected or unhealthy in the Akuity Cloud Console. Argo CD components may also fail to communicate with external services.

Logs show TLS verification failures such as:

tls: failed to verify certificate: x509: certificate signed by unknown authority

This typically occurs because SSL intercepts and re-signs TLS traffic using an enterprise CA that is not trusted by default inside Kubernetes containers.

Root Cause

There are two common issues when running behind Zscaler:

1. Missing Zscaler CA in containers
   Zscaler performs SSL inspection by intercepting HTTPS traffic and re-signing it with an organization-issued CA certificate.
   Since Akuity agents and Argo CD components act as HTTPS/TLS clients, they must trust this CA inside the pod.

2. SSL inspection interfering with Akuity traffic
   The Akuity agent establishes HTTPS/WSS tunnels to endpoints under: 

   *.cdsvcs.akuity.cloud


   SSL inspection can break these long-lived or tunneled connections unless explicitly bypassed.

Resolution

1. Inject the Zscaler CA Certificate into Akuity and Argo CD Pods

Use a Kustomization to:

• Create a ConfigMap containing the Zscaler CA certificate

• Mount that certificate into all relevant Deployments

• Place the certificate under /etc/ssl/certs, where it is automatically picked up by most base images

⚠️ Important
The certificate shown below is an example only. Replace it with your organization’s actual Zscaler root CA.

Below is the example shows that the akuity-agent and argocd-application-controller, but the certificate must be injected into all other relevant Deployments as well.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

generatorOptions:
  disableNameSuffixHash: true

configMapGenerator:
  - name: ca-pemstore
    namespace: akuity
    literals:
      - my-cert.crt=|
          -----BEGIN CERTIFICATE-----
          <YOUR ZSCALER ROOT CA HERE>
          -----END CERTIFICATE-----

patches:
  - target:
      kind: Deployment
      name: akuity-agent
    patch: |-
      - op: add
        path: /spec/template/spec/volumes/-
        value:
          name: ca-pemstore
          configMap:
            name: ca-pemstore
      - op: add
        path: /spec/template/spec/containers/0/volumeMounts/-
        value:
          name: ca-pemstore
          mountPath: /etc/ssl/certs/my-cert.crt
          subPath: my-cert.crt
          readOnly: true

  - target:
      kind: Deployment
      name: argocd-application-controller
    patch: |-
      - op: add
        path: /spec/template/spec/volumes/-
        value:
          name: ca-pemstore
          configMap:
            name: ca-pemstore
      - op: add
        path: /spec/template/spec/containers/0/volumeMounts/-
        value:
          name: ca-pemstore
          mountPath: /etc/ssl/certs/my-cert.crt
          subPath: my-cert.crt
          readOnly: true

2. Configure Zscaler SSL Inspection Bypass

In addition to trusting the CA, SSL inspection itself may still break agent connectivity.

Configure SSL inspection bypass rules in Zscaler for: 

This ensures:

• WebSocket (WSS) tunnels remain intact

• TLS is not re-signed or intercepted for Akuity control plane traffic