Developing Ansible Automation content on Apple Silicon

I love Linux. Always have, always will. I run Linux (mostly RHEL and Fedora) on a multitude of devices, from Raspberry Pi's to an old laptop functioning as a server, and VMs on my NAS.

But I have also ended up (long story) doing a lot of work on MacOS with Apple silicon, so I need a way to work on Ansible content from time to time, with a fast inner development loop. Translated: I want a fast way to quickly test my Ansible "code" on MacOS.

Roles and collections

I maintain a modest set of collections. Mostly for fun, but also to understand the workflow of my customers (I'm part of the Red Hat automation sales team). I have one for Gitea and act_runner, I have one for atuin and I am working on one for starship.

As you might see, those current collections and any future ones I write, are aimed at Linux machines, and RHEL in particular, either on x86_64 (amd64) or on aarch64 (arm64).

Writing roles and collections on MacOS

Writing a role or collection on MacOS is pretty easy. Whether you want to use VSCode, vim or anything else, everything is available on MacOS (as it is on Linux).

The tricky thing is to test the roles you write on MacOS on local Linux infrastructure, and in particular to test the roles on multiple architectures with a reasonable amount of speed. Especially that last part is hard. I have tried various things involving VMs and containers with qemu, and though that functionally works, it is sloooooow.

Rosetta

Then I read up on Rosetta. Rosetta is a binary translator that makes it very fast and very convenient to run x86_64 software on an m1, m2 or m3 aarch64 processor. It is FAST! It just wasn't very convenient to use when testing roles on multiple architectures in VMs on MacOS.

Setting up the VM

What I wanted to have was an aarch64 Linux VM on MacOS, that allowed me to run x86_64 containers for molecule testing with Rosetta as a translation layer with as little hassle as possible. (As said, I would have loved to use qemu for this, but it's just too slow.)

Basically, what you do is create a Lima VM on MacOS (that's the quickest), pass some flags during creation to have it mount the Rosetta translation binary, and configure the VM to invoke Rosetta whenever you execute an x86_64 binary.

So first, let's create the VM:

$ limactl create --containerd none --vm-type vz --rosetta --name fedora \
  --mount-writable template://fedora

Mounting and enabling Rosetta

Lima makes it easy to use Rosetta in the Linux VM. It's a matter of passing the --rosetta flag (as shown above) when creating a new machine. But in order to automatically invoke Rosetta when we call an x86_64 binary, we need to do a couple of things.

First of all, we need to drop a small config file ¹ as /usr/lib/binfmt.d/rosetta.conf. This file tells the operating system to call Rosetta when it tries to open a file with a certain set of magic bytes.

Then, we need to tweak the binfmt service itself a bit. When the binfmt service will first start, it does so before the Rosetta share is mounted. Therefore, we tell it to restart on failure after 5 seconds. Hardly a noticable delay and simple enough. This is achieved by a systemd drop-in:

[Service]
ExecStartPre=/usr/bin/test -f /mnt/lima-rosetta/rosetta
RestartSec=5
Restart=on-failure

SELinux

Because I run my tests on Fedora :heart, I needed to add some additional tasks to configure the SELinux policy for Rosetta. Rosetta does not run out of the box on a Fedora machine. That is because Rosetta needs to be mounted over NFS and it seems to need to be run from that location. I haven't researched this, but copying over the Rosetta binary to the VMs filesystem didn't seem to work at all.

Because the default SELinux policy disallows processes with init_t to execute files with an nfs_t label, we have to teach SELinux that this is OK behaviour for now.

The type enforcement file you will need for this, is as follows:

module rosetta 1.0;

require {
        type init_t;
        type nfs_t;
        class file { execute open read };
}

#============= init_t ==============

allow init_t nfs_t:file { execute open read };

Docker

Yes, that's right. At the moment, my Ansible dev workflow uses Docker CE. I'm planning on making it work with Docker, but in all honesty, it's still slightly easier to use Docker for molecule than it is to use podman. Especially when you plan to run molecule tests both locally, and as part of tests on GitHub.

Virtualenv

I like to keep anything I install through pip in virtualenvs. Makes it easier to test updates and makes it easier to work with different versions of Python packages for different use cases. My venv tool of choice is virtualenvwrapper.

I just want to get that installed, and create a virtualenv with all my Ansible dependencies in it, so I only have to log into my VM, run workon ansible and start doing whatever it is I need to do.

Making this quick and easy

I wrapped all of the above tasks in an Ansible playbook for my (and your) convenience. You can find it on GitHub. In order to use it, you have to do six things:

1. Install lima through your package manager of choice. I use brew, so I go brew install lima
2. Locally install Ansible on your Mac. I use a virtualenv for that, but since you are here, reading this, I assume you'll know how to do this
3. Create a Fedora VM called "fedora" with limactl.
4. Clone the repo from GitHub and cd to it
5. Copy inventory.tmpl to inventory and update it with your username
6. Run the playbook using ansible-playbook setup.yml

After that, you can navigate to the Ansible content you are working on on your Mac, and simply run limactl shell fedora to be dropped in the VM in the same directory. And after running workon ansible, you'll have all the Ansbile tools you need to start working on your collection or role.

Again, find the repo here!

Happy Ansible'ing!