It has a oxymoronic feel to it, the title, yet it seemingly is so that relative and absolute complexity downstream makes for a simpler upstream. Examples for this are Kubernetes and Docker, C and Python, and infrastructure surrounding HTTP servers and application servers.

I was watching the Kubernetes documentary (I and II) made by Honeypot and within they were stating that somewhere down the line a product more complex than Kubernetes will come along making everything else simpler again.

I have experienced this myself countless times as well, where if you want to make something simple for the user like with a press of a button you will get a bunch of things happen automatically, it requires quite a lot of complexity behind it. It requires scheduling, intricate and robust code that is fail safe and redundant, it requires talking to many services that each expose a simple API making the machinery behind that complex as well. It seemingly turns into a tree almost where the top of the tree is the end user clicking a button all the way down to the roots and it belies this hidden complexity as the user is only seeing the very top of the trees.

Kubernetes and Docker

So one of the earlier examples I gave was Kubernetes and Docker. Well Docker in and of itself is complex as it is (it hides away most virtualisation techniques for you to give you a fully running operating system running alongside your current operating system) yet when compared to Kubernetes it is quite simple. It has a simple CLI, a great developer experience (DX) and therefore it is pervasive in the cloud for a reason. Kubernetes is a beast, it has unclear documentation and the fact they use YAML for the config makes for a poor DX because as a developer it is quite difficult to see how these static YAML files produce a dynamic correctly configured infrastructure.

CloudFormation from AWS and HCL from Terraform suffer from the same problem

On the contrary there are some programmatic approaches to this like CDK, troposphere and Pulumi. These make it easier to see how things will be connected together and how the end result will look like.

It still requires you to know a lot about the respective cloud though

Not to digress to far, by comparison it is easy to see it is quite simple to reason about docker containers and images and not so simple to reason about StatefulSet, Deployments and Services and how they interact and intermingle. Let alone talk about Control Planes, Storage Providers and Ingress/Egress.

C and Python

C is the mother of many current programming languages we use. So also is it the base of Python, well CPython, and therefor the core of Python what actually gets run is quite complex. However the syntax and programming in Python itself is a breeze. So many concepts and things you do not have to know in order to be productive in Python all the while having C run behind all of that is wonderful.

In this case it is taking a language which is simple in syntax but complex in nature, C, and building on top of that this simple elegant language producing quite a complex codebase to manage that.

Infrastructure for HTTP servers

In this day and age running even a simple application is quite an ordeal. You need certificates to support HTTPS, you need to deploy your own code, which might be a full application or just some static files, then you need to configure an actual program to serve that content and make sure it all works. Then it needs to be further secure and taking care of all matter of things from OCSP, CSP, HSTS all the way to firewall rules, SELinux profiles and more.

Having said that though, since there is so much pushed into the web server that acts as a reverse proxy these days with SSL offloading makes for the fact that code coupled with the application server itself can become quite simple.

To that end there exists now this Java project called microhttp which ditches all those things and is just HTTP/1.0 compliant all written with the standard library of Java.

Conclusion

I think as we keep pushing the complexity further and further downstream it will spawn more and more simpler aspects behind it. So it will give more to learn ( as it will spawn more and more over time) however the chunks individually will be much easier to digest.

This does not mean by any means the computer / CPU itself will become easier, it is only getting more and more complex. It does however mean that we get “simple” instructions in the Assembly code that directly do very complex things like handling full matrix mathematical operations (TDPBSSD/TDPBSUD/TDPBUSD/TDPBUUD — Dot Product of Signed/Unsigned Bytes with Dword Accumulation) .It does all of that inside your CPU, that is amazing.

I hope to ride the wave a bit and learn these techs and at the same time think of ways to simplify and compress things into their more refined aspects.