David Currie

Returning to the 101 ways to create Kubernetes configuration theme, next up is ksonnet from the folks at Heptio. (I have no doubt that there are 101 ways to create Kubernetes configuration but I’m afraid I don’t really intend to cover all of them on this blog!) ksonnet has a different take yet again from Helm and kustomize. In many ways, it is more powerful than either of them but that power comes at the cost of a fairly steep learning curve.

The name is derived from Jsonnet, a data templating language that came out of Google back in 2014. Jsonnet essentially extends JSON with a scripting syntax that supports the definition of programming constructs such as variables, functions, and objects. The ‘Aha!’ moment for me with ksonnet was in realizing that it could be used as a simple template structure in much the same way as Helm. You start with some Kubernetes configuration in JSON format (and yq is your friend if you need to convert from YAML to JSON first) and from there you can extract parameters. I say ‘it could’ because you’d typically only take this approach if you were actually converting existing configuration but realizing this helped me get beyond some of the slightly strange syntax you see in generated files.

As usual, Homebrew is your starting point: brew install ksonnet/tap/ks. ksonnet has an understanding of the different environments to which an application is deployed and, when you issue ks init myapp, it takes the cluster that your current kube config is pointing at as the default environment (although you can override this with --context).

ksonnet then has the concept of ‘prototypes’ which are templates for generating particular types of application component when supplied with suitable parameters. These are provided by ‘packages’ which, in turn, come from a ‘registry’ stored on GitHub. Stealing from the tutorial, we can generate code for a simple deployment and service with the deployed-service prototype giving the image name and service type as parameters e.g.

ks generate deployed-service guestbook-ui \
  --image gcr.io/heptio-images/ks-guestbook-demo:0.1 \
  --type ClusterIP

At this point, we can use ks show default to return the YAML that would be generated or ks show apply to actually apply it to the default environment. I highly recommend doing the tutorial first and not the web-based tour as it shows you that you can get a long way with ksonnet without actually editing, or even looking at, any of the generated files. For example, you can use ks env add to create another environment and then ks param set to override the values of parameters for a particular environment as you might with Helm or kustomize.

Of course, the real power comes when you drop into the code and make use of ksonnet features like parts and modules to enable greater reuse of configuration in your application. At that point though, you really should take the time to learn jsonnet properly!

A short post for today but it relates to a tool that every Kubernetes user should have in their toolbox: kail. Although most users probably know that kubectl logs will, by default, show the logs for all containers in a pod and that it has --tail and -f options, fewer probably know that is has a -l option to select pods based on label. Kail takes tailing Kubernetes logs to a whole new level.

For Homebrew users, it’s available via brew install boz/repo/kail. When executed without any arguments it tails logs for all containers in the cluster which is probably not what you want unless your cluster is very quiet! There are, however, flags to let you filter not just on pod, container, and label, but also namespace, deployment, replica set, ingress, service, or node. Flags of the same type are ORed together, different flags are ANDed. And that’s pretty much all there is to it but anyone who finds themselves watching the logs of any moderately complex application will wonder how they lived without it!

Finally, I get to write that blog post on kustomize! kustomize is yet another tool attempting to solve the problem of how to make Kubernetes configuration re-usable. Unlike, say, Helm, kustomize allows configuration to be overridden at consumption time without necessarily having allowed for it when the configuration was originally produced. This is great if you are attempting to re-use someone else’s configuration. On the flip-side, you might prefer to use something like Helm if you actually want to limit the points of variability e.g. to ensure standardization across environments or applications.

You know the drill by now: the go binary CLI can be obtained via brew install kustomize. There is one main command and that is kustomize build. That expects to be pointed at a directory or URL containing a kustomization.yaml file. Running the command outputs the required Kubernetes resources to standard output where they can then be piped to kubectl if desired.

The kustomization.yaml can contain the following directives:

• `namespace` – to add a namespace to all the output resources
• `namePrefix` – to add a prefix to all the resource names
• `commonLabels` – to add a set of labels to all resources (and selectors)
• `commonAnnotations` – to add a set of annotations to all resources
• `resources` – an explicit list of YAML files to be customized
• `configMapGenerator` – to construct ConfigMaps on the fly
• `secretGenerator` – to construct Secrets via arbitrary commands
• `patches` – YAML files containing partial resource definitions to be overlayed on resources with matching names
• `patchesJson6902` – applies a JSON patch that can add or remove values
• `crds` – lists YAML files defining CRDs (so that, if their names are updated, resources using them are also updated)
• `vars` – used to define variables that reference resource/files for replacement in places that kustomize doesn’t handle automatically
• `imageTags` – updates the tag for images matching a given name

That’s a pretty comprehensive toolbox for manipulating configuration. The only directive I didn’t mention was bases with which you can build a hierarchy of customizations. The prototypical example given is of a base configuration with different customizations for each deployment environment. Note that you can have multiple bases, so aws-east-staging might extend both aws-east and staging.

One of the refreshing things about kustomize is that it explicitly calls out a set of features that it doesn’t intend to implement. This introduces the only other command that the CLI supports: kustomize edit. Given that one of the stated restrictions is that kustomize does not provide any mechanism for parameterising individual builds, the intent of this command is to allow you to script modifications to your kustomization.yaml prior to calling build.

It’s worth noting that kustomize can be used in combination with Helm. For example, you could run helm template and then use kustomize to make additional modifications that are not supported by the original chart. You can also use them in the reverse order. The Helmfile docs describe how to use Helmfile’s hooks to drive a script that will use kustomize to construct the required YAML, but then wrap it in a shell chart so that you get the benefit of Helm’s releases.