What's the go to for virtualization on apple silicon macs?

[azonenberg@ioc.exchange]

What's the go to for virtualization on apple silicon macs?

Requirements:
* Free or one time purchase, not subscriptionware
* Can run macos guests with full metal support
* CLI for spawning, terminating, snapshotting, and reverting

Stretch goals:
* Can run arm64 Linux guests with vulkan support

[azonenberg@ioc.exchange]

It looks like the players so far are vmware fusion, UTM, and parallels

[petrillic@hachyderm.io]

@azonenberg I use UTM mostly.

[azonenberg@ioc.exchange]

@petrillic do you get accelerated metal graphics in the guest?

[petrillic@hachyderm.io]

@azonenberg I have never done graphics in the VMs, it's always CLI. But it's built on qemu, and I know it does support OpenGL/Metal using the ANGLE project. But I don't think Vulkan

[azonenberg@ioc.exchange]

@petrillic if the guest has metal i can get vulkan via MoltenVK

[dngrs@chaos.social]

@azonenberg tangent, afaik NOT meeting the GPU criteria, but might be interesting to know for related projects:

GitHub - apple/container: A tool for creating and running Linux containers using lightweight virtual machines on a Mac. It is written in Swift, and optimized for Apple silicon.

A tool for creating and running Linux containers using lightweight virtual machines on a Mac. It is written in Swift, and optimized for Apple silicon. - GitHub - apple/container: A tool for creating and running Linux containers using lightweight virtual machines on a Mac. It is written in Swift, and optimized for Apple silicon.

GitHub (github.com)

[azonenberg@ioc.exchange]

Quick summary from preliminary doc reading having not tried any of them yet:

* VMware does not appear to support virtualizing macos apple silicon guests

* Parallels appears to be subscriptionware but will probably do what I want if I buy the pro edition (seems the lower tier doesn't have the CLI)

* UTM is free and seems to support MacOS guests, so I'm gonna try to install it and see what happens

[pmdj@mstdn.social]

@azonenberg FWIW, I use the super-basic ‘macosvm’ CLI* for spawning my ephemeral arm64 macOS CI runners.
All the options you listed and this one just use Apple’s own Virtualization.framework for virtualising arm64 macOS. So feature wise they have parity for this specific purpose. Other guest OSes are usually implemented via the lower level Hypervisor.framework so will vary greatly in features.

Metal works in VZ.fwk, but only baseline features.

* https://github.com/s-u/macosvm

[natevw@toot.cafe]

@azonenberg my understanding is that everything that supports macOS guests these days is based on (Apple's own) same Virtualization framework and thus all inherit the same constraints re. GPU (and USB) passthrough.

[edit to clarify: UTM which I like/recommend does use *multiple* back ends and so for other e.g. Linux/Windows guests it can use Qemu which itself has multiple potential backends of its own which are less limited. other hypervisors probably similar story but all converging for macOS]

[pmdj@mstdn.social]

@azonenberg As an aside, QEMU contains the only non-VZ.fwk implementation capable of running arm64 macOS guests. So that offers the same features as other QEMU backed VMs.
But we never got beyond running newer than macOS12 guests, and the PV graphics code went stale for 15.4(?)+ hosts so it’s currently disabled in the build by default. (I no longer have a source of funding for macOS virtualisation work, and I can’t really afford to spend significant time on it otherwise.)

[petersommerlad@mastodon.social]

virtualbox?

[azonenberg@ioc.exchange]

@rfc6919 if so then that's fine. i'll have to check but utm is free so i'll try it first

[toader@fosstodon.org]

@azonenberg Definitely try UTM first. Last time I tried macOS guests, tjey looked GPU accelerated, but didn’t dig in the features.

Linux guests have the usual QEMU backends, some of which are GPU enabled, but might be beta-ish.

Bonus points for linux guests, if you don’t require QEMU for them: you can use rosetta, so you can run x86 apps/libs also in Linux.

There is also libkrun which advertises gpu accelerated containers on macOS via venus. Haven’t tried this tho.

[azonenberg@ioc.exchange]

@toader I already have a big iron x86 machine to run x86 stuff. The only reason I have the mac mini is to test on MacOS.

But I might try qemu with Linux guests since we do not currently have an ARM64 Linux test environment and it'll probably run faster virtualized than software emulated on x86.

[toader@fosstodon.org]

@azonenberg Definitely give it a try, never not a good reason to also have ARM64 support.

Regarding performance, my M4 Mini is giving a lot of Ryzen boxes a run for their money in Vivado synthesis (in a Linux VM, ‘emulating’ x86 via Rosetta). The single core perf is that good and virtualization is seamless.

UTM is stable, and also has a server for managing VMs via an API.

[azonenberg@ioc.exchange]

@toader The mac mini has 16GB of RAM and the Xeon box has 512. I'm not running any VMs on the mac that don't actually *need* apple silicon.

But yeah, one mac runner and one Debian arm64 runner or something, taking turns because I probably won't have the RAM to do both simultaneously, is probably doable.

[hywan@floss.social]

@azonenberg @whitequark OrbStack: easy, intuitive, just works, https://orbstack.dev/

[whitequark@social.treehouse.systems]

@hywan @azonenberg that runs macos guests?

[hywan@floss.social]

@whitequark @azonenberg Probably not.