Introducing Kubie

2020-03-31

Kubie is a tool that I've created to improve my day to day life at work. I work with a lot of kubernetes clusters: I have around two dozen clusters in my config files. I also happen to multitask a lot. I use the i3 window manager to keep my work organized, often having 7 or 8 different workspaces active at once.

I kept being bothered by the fact that kubectx and kubens modify a global config file. That is, when you change your shell's current context or namespace, it also affects all other shells.

Thus, the primary purpose of kubie is to isolate shells from each other. It does so by never mutating the global config files. When you first enter a context, a temporary file which contains the context's config is created, a bash shell is spawned, and the KUBECONFIG environment variable is set to this temporary file. When you change the namespace, the change is performed in the temporary file, meaning that it does not affect other shells.

There are other cool features in kubie beyond isolation. They are:

Note that kubie is not meant to replace kubectl. It's mean to be a companion tool to kubectl that simplifies working with multiple clusters and maintaining their config files.

If you're already sold on kubie and don't care about those other features to you can skip to the Getting started section right away.

Shell prompt modification

Since it spawns a new shell, kubie is able to control the PS1 variable inside of that shell. You do not have to modify your .bashrc to display inside of which context and namespace you are, kubie does that for you.

kubie is also designed to play nice with modified prompts. All it does is prepend something to your current PS1, it does not enforce any particular style.

So if my prompt normally looks like simon@debian:~/projects/homepage/blog$, inside of a kubie shell, it will look like [context|namespace] simon@debian:~/projects/homepage/blog$. If my prompt is normally just $, then it will look like [context|namespace] $ inside of a kubie shell.

For now it's not possible to customize the prepended string, but nothing prevents it from being customizable in the future.

Split configuration files

As I've mentionned before, I work with a ton of kubernetes clusters. Keeping the information for all these clusters in a single YAML config file is an impossible to manage mess. So I've decided to add built-in support for splitting up configuration files in kubie.

So by default, kubie will search for kubernetes config files using these globs:

~/.kube/config
~/.kube/*.yml
~/.kube/*.yaml
~/.kube/configs/*.yml
~/.kube/configs/*.yaml
~/.kube/kubie/*.yml
~/.kube/kubie/*.yaml

These paths can be configured to anything you like. You can find more info about kubie's own config file on the Github repository.

kubie also supports name disambiguation. That is, when kubie searches the cluster and user associated with a context it will only look for clusters and users within the same config file. This allows clusters and users to use the same cluster name and user name in each file while not confusing any of the values. So you can name every cluster "cluster" and every user "user" if you want, as long as they're all in separate files, there's no issues.

This has proved to work very well for my colleagues and I. It could be possible to change this behavior to prefer items within the same file, but still look at what's in other files. At the moment this isn't implemented.

So how should each kubernetes config file look like? Ideally you want to put one cluster and one user per file and then have one or more contexts. For instance:

# File: ~/.kube/configs/context-1234.yaml
clusters:
  - name: cluster-1234
    cluster:
      insecure-skip-tls-verify: true
      server: https://1.2.3.4
users:
  - name: user-1234
    user:
      username: admin
      password: hunter2
contexts:
  - name: context-1234
    context:
      cluster: cluster-1234
      user: user-1234
      namespace: api

Note that it is not necessary to identity the current context with the current-context property as kubie does not rely on the config files to identity the context currently in use.

kubie exec

kubie exec is a subcommand that allows you to run commands inside of a context, a bit like kubectl exec allows you to run a command inside a pod.

For instance kubie exec <context> <namespace> kubectl get pods will list all the pods inside of the given namespace and the given context. It can be very useful for scripts that need to run things in a specific context and namespace.

There isn't any filtering done on the command that is given, so your imagination is the limit. You can run kubectl exec inside of kubie exec if you'd like. For instance kubie exec production default -- kubectl exec psql -c 'drop database app' will drop your production database and make you unemployed.

kubie exec also supports wildcard matching on contexts, so if you have a context named api-dev, a context named api-staging, and a context named api-prod, you can do kubectl exec api-* ... to run the given command in each of the matched context. For instance you could do kubectl exec api-* default helm ls to list all the charts installed with helm in each of the matched contexts.

Getting started

Getting started with kubie should be really simple. By default kubie will read the information available in ~/.kube/config, so you should see your list of contexts even if you haven't split your kubernetes config files.

Check the repository's README to see how kubie can be installed. Note that improvements to the build system and distribution system are in the plans, see the Future plans section.

To enter a context, use kubie ctx <context>. If you have fzf installed, which is a command line fuzzy finder, using kubie ctx should present you with a selectable and filterable menu of contexts. If you do not have fzf installed, kubie will print a list of contexts.

To enter a namespace, use kubie ns <namespace>. Once again if you have fzf, kubie ns will show you a menu. Using kubie ns without fzf installed will print a list of namespaces.

That's pretty much all you need to know to get started with kubie. Check the README for more information about the other commands and features.

Future plans

kubie itself is stable and works very well. The ecosystem around it still needs work however. Here are the main future plans for kubie as of today:

Binaries and distribution: the top priority for me at the moment is to build pipelines that can automatically build and push binaries to GitHub releases. I'm planning on building binaries for Mac and Linux. Windows support is not planned at the moment. kubie relies on bash to exist on the machine. It would probably work in a cygwin / Git Bash for Windows environment, but I haven't tested it.
Auto update: once the pipelines are built, it will become possible to auto-update kubie using GitHub releases. The feature would probably look like what rustup does, kubie self-update.
zsh and fish support: currently kubie only supports spawning a bash shell. It should at the very least support zsh in the future.
Vault integration: this feature is more of a team feature, but essentially it would be the ability for kubie to download kubernetes config files from a secrets server, such as Hashicorp's vault. That way you can synchronize your configs on a regular basis without having to do it manually.