Helm deploy succeeded. Config never applied.

kubernetes helm helmfile devops platform-engineering

We added a new API route to the gateway service. The helmfile pipeline ran, every stage went green, and the deploy completed with no errors.

The route returned 404.

A port-forward and a curl confirmed it: the new route definition was not on the live gateway. The pod was running the config from the last restart, not the one we had just deployed. We had updated a ConfigMap. The pod had no idea.

What made this harder to spot: it had worked before. Two months earlier, a similar config change had gone live without incident. What we did not know then is that the change happened to also touch a volume mountPath, which caused Kubernetes to roll the pod as a side effect. The config update looked causal. It was coincidence.

Why ConfigMaps do not trigger restarts

Kubernetes rolls a pod when its pod template spec changes. A ConfigMap is a separate object in the Kubernetes API. Updating one does not mutate the pod template. The pod continues running with whatever was mounted at its last start.

This is by design. Kubernetes deliberately separates “the data changed” from “the consumer needs to restart.” For apps with in-process file watchers, this is the right behavior: they can reload config without a pod restart. For apps that read config files at startup and hold them in memory, it is a trap.

The gateway service falls into the second category. It reads API definitions from mounted files when it starts. There is no file watcher. A ConfigMap update lands on disk and stays there, invisible to the running process, until something restarts the pod.

The standard fix: checksum annotation

The canonical solution is to add a checksum/config annotation to the Deployment’s pod template, derived from the ConfigMap content:

# templates/deployment.yaml — standard single-chart pattern
spec:
  template:
    metadata:
      annotations:
        checksum/config: {{ include "gateway.configmap" . | sha256sum | quote }}

When the ConfigMap content changes, the annotation value changes. The annotation lives inside spec.template.metadata, which is part of the pod spec. Kubernetes sees the pod template has changed and triggers a rolling restart. The annotation value is meaningless as metadata; its only job is to carry the change signal into the pod spec.

This pattern appears in the Helm docs and most writeups on forcing pod restarts on ConfigMap changes. It works well in a single Helm chart where include "gateway.configmap" can reach the named template.

Why it breaks across releases

Our setup has two separate Helm releases: one that owns the gateway Deployment and one that generates the API-definition ConfigMaps from individual definition files.

include is scoped to the current chart. You cannot call include "other-release.configmap" from a Deployment template; the named template does not exist in that chart’s rendering context. The ConfigMap content is not available to the Deployment chart at render time.

This is the part most checksum-annotation writeups skip. They assume a monolithic chart where the Deployment and ConfigMap share a template namespace. The moment you split them into separate releases, which is common for dynamically-generated config, the standard pattern has no way to reach across.

The helmfile solution

helmfile values files support Go templating with a set of functions beyond standard Helm. Two of them solve this problem directly.

readFile reads a file’s raw content and returns it as a string. readDirEntries returns the entries in a directory. Both resolve paths relative to the values file’s own directory, not the repo root. That last detail matters.

In the gateway release’s values file, we read every source file that feeds the ConfigMaps, concatenate the content, hash it, and expose the result as a values key:

# helmfile-values/gateway/values.yaml.gotmpl
{{- $apiDir := "./apis" -}}
{{- $content := "" -}}
{{- range readDirEntries $apiDir -}}
{{-   $content = cat $content (readFile (printf "%s/%s" $apiDir .Name)) -}}
{{- end -}}
configChecksum: {{ $content | sha256sum | quote }}

The ./apis path resolves relative to the values file’s location. If values.yaml.gotmpl lives at helmfile-values/gateway/values.yaml.gotmpl, then ./apis must be helmfile-values/gateway/apis/ on disk. Get this wrong and readDirEntries returns an empty list, $content stays empty, and the hash is constant. The annotation renders but never changes regardless of what you put in the source files.

The Deployment template then reads the pre-computed hash from values:

# gateway chart — templates/deployment.yaml
spec:
  template:
    metadata:
      annotations:
        checksum/config: {{ .Values.configChecksum }}

The key insight: the hash computation happens in the helmfile values layer, before any chart-scoped templating runs. The Deployment chart does not need to know about the ConfigMap chart. The two releases are decoupled; the checksum bridges them through the values interface.

Verifying it works

helmfile template renders the Deployment YAML with the annotation populated. Edit any file in the APIs directory, re-run helmfile template, and confirm the hash in the annotation changes. If it does not change, the path is wrong. Add {{ $content | len }} to the values template to confirm content is being read at all.

helmfile diff surfaces the annotation update as a clean pod-template change: just the annotation value, nothing else. After apply, a port-forward and a curl on the new route confirms the config is live.

What to take from this

Multi-release architectures break single-release assumptions. The checksum annotation is well-documented for single charts; the cross-release variant almost never appears in official docs or blog posts.

helmfile’s readFile and readDirEntries are the right escape hatch because they operate at the values layer, before any chart-scoped template context runs. This makes them available to any release that needs to hash content from another release’s source files.

The relative-path behavior is the one place the abstraction leaks. Document the path mapping in your repo’s values file with a comment once, and it stays solved.

The same pattern applies to any service that reads config from mounted files at startup: message brokers, proxies, certificate bundles, policy engines. If the service does not watch for file changes at runtime, a ConfigMap update is silent until something restarts the pod. The checksum annotation makes that restart automatic and deterministic.