My Experience with Kubeflow
This is a short writeup/log of my experience with Kubeflow.
UNDER CONSTRUCTION
What did I learn?
I learned a lot about Docker, Kubernetes, and containerization as a whole.
Command Line Tools
I got comfortable building command-line tools in python. I did go with the click library to do it though, so maybe that’s not a great thing.
Building a tool to run from the command line ensures it’s portable. Kubeflow required it as each stage was a container that was running command-line tools. This has already paid huge dividends in my other projects.
Kubernetes-ish
I would be lying if I said I “learned” kubernetes. While I know signficantly more than I did before, I still do not feel comfortable with nor do i feel close to getting comfortable with it.
Private Docker Registry
I learned how to host my own docker registry to support my pipelines. This avenue was explored due to the limitations on private images on Docker Hub. I learned to secure a registry or simply tell each docker host to allow insecure registries.
I did get lazy though. It’s a lot of work to manage a private docker registry, so I did end up using docker hub since it’s the out-of-the-box default. There wasn’t even anything to protect in my images. I just like spinning up my own resources when I can.TODO
#
