Using Minikube On Fedora With The Registry Addon
Posted on wo 05 februari 2020 in dev
Table of Contents
Update (March 6, 2020)
I have updated the below description for minikube 1.7.3, which is a little different than the 1.6.2 version that I originally wrote this tutorial for.
The main difference is in what minikube config is done (I'm adding a vm-driver config
option, and an insecure-registry option), though I'm not 100% if this is working,
because I'm also still explicitly using the --insecure-registry
flag and enabling the
registry addon. Without these, pushing an image into the registry (or pulling from it)
for a deployment, does not work.
Enabling the registry addon needs to happen after starting the minikube VM now though, and not before.
Setting up minikube on Fedora 31
The article on setting up OpenShift 4.3 on libvirt VMs is not quite done yet. In the meantime though, I've been playing around with minikube as well. Just to get a feel of plain Kubernetes without the productive / useful abstractions OpenShift uses (OpenShift being certified Kubernetes, btw).
Setting up a minikube VM
Downloading the required binaries
Setting up minikube on Fedora is really, really easy. It's not packaged as an RPM, but you only have to go here and download the latest release. I prefer to stick with stable, which is v1.7.3 at the moment of writing this.
Together with the main minikube binary, you need to download the
docker-machine-driver-kvm2
binary. Drop both that file and the minikube
binary itself into your PATH
and you are ready to go.
(There is an RPM available as well, but I use the binary downloads.)
Configuring minikube
Minikube should work fine without any additional configuration at this point, but I usually make some changes.
The first thing I like to do, is increase the default memory. According to the doc, by default, minikube only allocates 2GiB of RAM, which is next to nothing. I increase it to at least 10GiB instead.
I also like to switch the amount of vCPUs to at least 4, but sometimes more, depending on what I'm playing around with (I have a ton of CPUs in my machine anyway). The default is 2 vCPUs; if that's enough for you, leave it as is.
Let's also increase the size of the disk to 30GiB so we can store some
images. 20GiB is the default here, and it will be created as a thin
provisioned, raw disk image in ~/.minikube/machines
by default.
Finally, in order to make sure we use the container runtime that aligns best
to Kubernetes itself, we set the container-runtime configuration property to crio
.
$ minikube config set memory 10240
$ minikube config set cpus 4
$ minikube config set disk-size 30000
$ minikube config set container-runtime crio
$ minikube config set vm-driver kvm2
$ minikube config set insecure-registry 192.168.39.0/24
Starting the minikube VM
This should be a fairly short paragraph, but there's one thing to talk about here. Minikube allows you to run different instances of minikube in different profiles. Profiles are basically named instances of minikube.
For example, you can have a minikube instance to play around with to learn
Kubernetes itself, and you can have one to - I don't know - test your app in.
The first could be called k8s-test
, the second one myapp
.
In order to start a named minikube instance, we run it with the -p
option:
$ minikube start -p mytest
Start a minikube instance for real, and let's push some images into the
registry living on it! Make sure to use the --insecure-registry
option
like below. You'll need that to have minikube pull from it!
$ minikube start -p mytest --insecure-registry "192.168.39.0/24"
If you minikube stop -p mytest
, in order to start your minikube VM again, you will
need to use the --insecure-registry
flag again!
Enabling the registry addon
I want to be able to push images into minikube, so I need to run a registry in it. We can get a list of available addons to minikube by running:
$ minikube addons list
This will show you all the available options. To enable the registry addon, we only need to run while our mytest minikube VM is running (which is different from previous versions of minikube!):
$ minikube addons enable registry -p mytest
That's all, we good to go!
Pushing images into the registry and running them!
I want to be able to build images of my app locally, then push them into the
minikube registry and run them from there. The registry on minikube does not
have TLS enabled. To get images in there, for me the easiest way to do that
was to wield skopeo
.
First, let's see what images we have, locally:
$ podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.redhat.io/rhel8/nodejs-10 latest 72b60f3a377b 6 weeks ago 788 MB
registry.redhat.io/openjdk/openjdk-11-rhel7 latest 9a6cd285b6c8 6 weeks ago 510 MB
registry.access.redhat.com/redhat-openjdk-18/openjdk18-openshift latest 963af2cdd61f 6 weeks ago 491 MB
registry.redhat.io/rhel8/postgresql-10 latest 58950bf238bf 8 weeks ago 407 MB
registry.access.redhat.com/ubi8/ubi latest 096cae65a207 8 weeks ago 239 MB
registry.redhat.io/ubi8/ubi latest 096cae65a207 8 weeks ago 239 MB
registry.access.redhat.com/ubi8/ubi 8.1 096cae65a207 8 weeks ago 239 MB
registry.access.redhat.com/ubi8/ubi-minimal latest 469119976c56 3 months ago 107 MB
registry.fedoraproject.org/fedora-minimal latest 7bb5a60e8a78 3 months ago 167 MB
registry.fedoraproject.org/f29/httpd latest 25c76f9dcdb5 9 months ago 482 MB
As an example, let's push the fedora-minimal
image into our registry:
$ skopeo copy --dest-tls-verify=false docker://registry.fedoraproject.org/f29/httpd docker://$(minikube ip -p mytest):5000/f29/httpd
Getting image source signatures
Copying blob 7692efc5f81c skipped: already exists
Copying blob d77ff9f653ce skipped: already exists
Copying blob aaf5ad2e1aa3 skipped: already exists
Copying config 25c76f9dcd done
Writing manifest to image destination
Storing signatures
And that's it. We can now start that container.
Starting the container
We point the deployment to the local registry and everything from that point on is automatic:
$ kubectl run apache --image=$(minikube -p mytest ip):5000/f29/httpd
Let's check what's going on:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
apache-f698479d9-ltqvf 1/1 Running 0 11m
$ kubectl logs apache-f698479d9-ltqv
=> sourcing 10-set-mpm.sh ...
=> sourcing 20-copy-config.sh ...
=> sourcing 40-ssl-certs.sh ...
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.1.0.18. Set the 'ServerName' directive globally to suppress this message
[Wed Feb 05 09:30:46.294280 2020] [ssl:warn] [pid 1:tid 140049696796032] AH01882: Init: this version of mod_ssl was compiled against a newer library (OpenSSL 1.1.1b FIPS 26 Feb 2019, version currently loaded is OpenSSL 1.1.1 FIPS 11 Sep 2018) - may result in undefined or erroneous behavior
[Wed Feb 05 09:30:46.294874 2020] [ssl:warn] [pid 1:tid 140049696796032] AH01909: 10.1.0.18:8443:0 server certificate does NOT include an ID which matches the server name
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.1.0.18. Set the 'ServerName' directive globally to suppress this message
[Wed Feb 05 09:30:46.352319 2020] [ssl:warn] [pid 1:tid 140049696796032] AH01882: Init: this version of mod_ssl was compiled against a newer library (OpenSSL 1.1.1b FIPS 26 Feb 2019, version currently loaded is OpenSSL 1.1.1 FIPS 11 Sep 2018) - may result in undefined or erroneous behavior
[Wed Feb 05 09:30:46.352919 2020] [ssl:warn] [pid 1:tid 140049696796032] AH01909: 10.1.0.18:8443:0 server certificate does NOT include an ID which matches the server name
[Wed Feb 05 09:30:46.353031 2020] [lbmethod_heartbeat:notice] [pid 1:tid 140049696796032] AH02282: No slotmem from mod_heartmonitor
[Wed Feb 05 09:30:46.354288 2020] [mpm_event:notice] [pid 1:tid 140049696796032] AH00489: Apache/2.4.39 (Fedora) OpenSSL/1.1.1 configured -- resuming normal operations
[Wed Feb 05 09:30:46.354302 2020] [core:notice] [pid 1:tid 140049696796032] AH00094: Command line: 'httpd -D FOREGROUND'
Excellent! Happy containering from this point on!