Well, 2024 was a busy one for me!

I was on holiday recently and try to avoid deep diving while on family time, so I casually browse and if I find something I think I’d like to read in greater depth at a later point in time, I save it.

In this case I happened across a great post on HackerNews that talked around a problem I’ve experienced - and by problem I mean a preference issue, since it’s not technically a problem per se - with the Kubernetes CLI tool, kubectl.

I’ll summarise it briefly, but here’s the original post for the curious: Avoid kubectl’s global state.

Context⌗

My day job has me working in a Kubernetes cluster-rich environment. Sometimes I’d like to be looking at one cluster and comparing it to another, perhaps for checking for differences or for similarities (and as an example I’m currently migrating workloads across both deployment methods and clusters).

I use the excellent kubectl augmentation tool kubectx (available here) to toggle between cluster contexts, which allows me to abbreviate commands;

kubectl --context clusterA get nodes
kubectl --context clusterA get pods

becomes

kubectx clusterA
kubectl get nodes
kubectl get pods

The problem⌗

It’s another command to run to switch to the context, but once there it makes life quicker. However, if I were to pop open a second terminal and activate my work kubectl config, it automatically picks up the state as determined by the commands run in the first terminal, which change the stored state in the kubeconfig file.

Great if you’re working in one place, but if working across more than one cluster it could lead to playing context-pong, or worse, a mistake of actual damage.

I’ve done it before; I have Bash aliases set up to separate my homelab contexts and my work contexts into totally separate config files for this reason. It looks something like this:

alias kube-personal='export KUBECONFIG=~/.kube/kubeconfig-homelab'
alias kube-work='export KUBECONFIG=~/.kube/kubeconfig-work'

Combined with a default config file that is empty, this ensures that I have to make an intentional choice to be working in either of those contexts.

My solution: duplication⌗

The solution suggested in the article was to create a new kubeconfig file for each terminal; this didn’t appeal to me for the simple reason that my work has an AWS SSO integration that controls access to the clusters, and so would require additional time to reauthenticate in each terminal opened. Such tedium.

Instead, I mushed together some of the solution and came up with this one-liner to get the best of both worlds: my preconfigured (and authenticated as of that day - boo for 8 hour token expiry) kubeconfig could be carried over to new terminals, but I would also be able to access that config independently in each new terminal.

And here it is (with \’s for legibility):

alias kube-work='NEWCONFIG=~/.kube/work-temp-$(date +%s%3N) && \
    cp ~/.kube/kubeconfig-work $NEWCONFIG && \
    export KUBECONFIG=$NEWCONFIG && \
    trap "rm -f $NEWCONFIG" EXIT'

Works like a charm.