Kustomizing Kubernetes Konfiguration

Finally, I get to write that blog post on kustomize! kustomize is yet another tool attempting to solve the problem of how to make Kubernetes configuration re-usable. Unlike, say, Helm, kustomize allows configuration to be overridden at consumption time without necessarily having allowed for it when the configuration was originally produced. This is great if you are attempting to re-use someone else’s configuration. On the flip-side, you might prefer to use something like Helm if you actually want to limit the points of variability e.g. to ensure standardization across environments or applications.

You know the drill by now: the go binary CLI can be obtained via brew install kustomize. There is one main command and that is kustomize build. That expects to be pointed at a directory or URL containing a kustomization.yaml file. Running the command outputs the required Kubernetes resources to standard output where they can then be piped to kubectl if desired.

The kustomization.yaml can contain the following directives:

  • namespace – to add a namespace to all the output resources
  • namePrefix – to add a prefix to all the resource names
  • commonLabels – to add a set of labels to all resources (and selectors)
  • commonAnnotations – to add a set of annotations to all resources
  • resources – an explicit list of YAML files to be customized
  • configMapGenerator – to construct ConfigMaps on the fly
  • secretGenerator – to construct Secrets via arbitrary commands
  • patches – YAML files containing partial resource definitions to be overlayed on resources with matching names
  • patchesJson6902 – applies a JSON patch that can add or remove values
  • crds – lists YAML files defining CRDs (so that, if their names are updated, resources using them are also updated)
  • vars – used to define variables that reference resource/files for replacement in places that kustomize doesn’t handle automatically
  • imageTags – updates the tag for images matching a given name

That’s a pretty comprehensive toolbox for manipulating configuration. The only directive I didn’t mention was bases with which you can build a hierarchy of customizations. The prototypical example given is of a base configuration with different customizations for each deployment environment. Note that you can have multiple bases, so aws-east-staging might extend both aws-east and staging.

One of the refreshing things about kustomize is that it explicitly calls out a set of features that it doesn’t intend to implement. This introduces the only other command that the CLI supports: kustomize edit. Given that one of the stated restrictions is that kustomize does not provide any mechanism for parameterising individual builds, the intent of this command is to allow you to script modifications to your kustomization.yaml prior to calling build.

It’s worth noting that kustomize can be used in combination with Helm. For example, you could run helm template and then use kustomize to make additional modifications that are not supported by the original chart. You can also use them in the reverse order. The Helmfile docs describe how to use Helmfile’s hooks to drive a script that will use kustomize to construct the required YAML, but then wrap it in a shell chart so that you get the benefit of Helm’s releases.

Leave a Reply