Exploring kubernetes the hard way
So I’ve been in the AWS cloud space for a long time which has been great as they really have phenomenal cloud offering, but in that time Kubernetes has been steadily gaining speed as an alternative hybrid approach to cloud computing. And while I’ve read some things and worked on containers running in K8’s I’ve never really had an in depth understanding of the cluster management system. So I decided to fix that by doing the "Kubernetes the Hard Way" tutorial. It’s been great, while not a whole lot of it is new, it really is great practice and a very good end to end, "secure", setup walk through. So these are some take a ways I have from the experience.
While the tutorial says "no scripts", that’s not exactly accurate. It will have you write your own setup scripts and
bash is the language used to explain those operations. While the code display pieces on the tutorial have an "easy button" to copy the code and paste it in your terminal there are two main reasons not to do this:
- Don’t copy and paste code from the web in your terminal! There are well documented attack vectors that can compromise your entire system by doing this.
- The idea is to get a non-shallow understanding of Kubernetes, copying the code and writing it your way adds muscle memory to the exercise.
Additionally unless you are a super human typist there are the inevitable typos, bugs and such that actually help to learn in depth how to diagnose problems in Kubernetes and fix them. This is really what I find most useful about doing it the hard way to really learn. For instance, the tutorial assumes using tmux in parallel manner across 3 controllers when initializing
etcd on the controllers, which I didn’t do. Through which I found out that the second controller must be initialized within the timeout period of the daemon start on the first or you will get a timeout error (after those first two are initialized this is no longer an issue). I would likely have never learned this by using scripted setups for K8’s clusters.
Or like when I typo’d a cert location as
/etc/kubernetes.pem instead of
/etc/etcd/kubernetes.pem and I learned I’ve been absolutely spoiled by Rust’s detailed and helpful compiler error messages. The error message was something like
ERROR: systemd returned a status of "error code" returned. I know what you are thinking, "
error code" should be more than enough for you to know what went wrong. Unfortunately I needed a bit more detail to figure out the problem so a bit of research showed me the command
journalctl -ocat -b -u etcd
Where "etcd" is really whatever your
systemd daemon name is. I think I’m going to
alias this to
doh in my shell for future reference. I know
journalctl but the argument soup is a super useful combo for working with
systemd daemons, but one which I’m not sure I’ve ever used or have forgotten. So learning/re-learning it because I’m doing this the "hard way" has been really great. I’d highly recommend this tutorial if you’d like to learn hands on about Kubernetes.