link =/= sync: how i got reliable stage visuals running on linux from Ableton [and possibly other DAWs] (crossplatform)
-
link =/= sync: how i got reliable stage visuals running on linux from Ableton [and possibly other DAWs] (crossplatform)
more technical explanation below
-
link =/= sync: how i got reliable stage visuals running on linux from Ableton [and possibly other DAWs] (crossplatform)
more technical explanation below
this is not “sync” in the way ableton markets sync using "Link"
ableton is running on macos or windows, doing what it normally does, but instead of trying to convince another machine to “follow along musically”, it just emits actual midi realtime messages and position data over rtp midi. linux picks that up, interprets it, and drives video playback directly with MPV.the architecture is stupid simple once you stop thinking in terms of link. ableton sends midi into an rtp midi session (apple midi over network, whatever you want to call it, it’s all the same protocol over udp). on macos that’s built in, on windows it’s that tobias erichsen "RTPmidi" driver, and on linux you can run rtpmidid which basically pretends to be a native participant in that ecosystem. rtpmidid listens on the network, joins the session via avahi, and then exposes whatever it receives as an alsa sequencer port. so from the linux side it just looks like a normal midi input called “rtpmidid: AbletonSync”
rtpmidid injects events into alsa, and then a tiny python script reads from that port using mido + python-rtmidi. there’s no heavy timing logic needed. it’s literally just reacting to incoming midi messages as they arrive. the important part is what messages you react to, because if you naively trigger on “note on” or something you’ll get garbage behavior and possibly not even a trigger.
-
this is not “sync” in the way ableton markets sync using "Link"
ableton is running on macos or windows, doing what it normally does, but instead of trying to convince another machine to “follow along musically”, it just emits actual midi realtime messages and position data over rtp midi. linux picks that up, interprets it, and drives video playback directly with MPV.the architecture is stupid simple once you stop thinking in terms of link. ableton sends midi into an rtp midi session (apple midi over network, whatever you want to call it, it’s all the same protocol over udp). on macos that’s built in, on windows it’s that tobias erichsen "RTPmidi" driver, and on linux you can run rtpmidid which basically pretends to be a native participant in that ecosystem. rtpmidid listens on the network, joins the session via avahi, and then exposes whatever it receives as an alsa sequencer port. so from the linux side it just looks like a normal midi input called “rtpmidid: AbletonSync”
rtpmidid injects events into alsa, and then a tiny python script reads from that port using mido + python-rtmidi. there’s no heavy timing logic needed. it’s literally just reacting to incoming midi messages as they arrive. the important part is what messages you react to, because if you naively trigger on “note on” or something you’ll get garbage behavior and possibly not even a trigger.
the correct signals to use are midi realtime + song position pointer. ableton sends spp messages that tell you where the transport is, and realtime messages like start/stop/continue that tell you what it’s doing. the trick is that spp by itself is not a play event, it’s just a seek. so what you do is you arm when you see “songpos == 0” (meaning the transport is at the beginning), and then you actually trigger playback when you receive a “start” message while armed. that way you only fire when the sound engineer actually presses play from the beginning, not when they scrub around or restart mid timeline.
on the video side it’s mpv, because mpv is basically perfect for this. you start it once, paused, with a unix ipc socket exposed. something like “mpv --pause --fullscreen --input-ipc-server=/tmp/mpv-show.sock video.mp4”. that socket is just a json command interface. the python script connects to it and sends commands like “set pause false” or “seek 0 absolute”. this is way faster and more reliable than spawning a new player every time, and it removes startup latency almost entirely (~0.1ms latency at most, wow!).
everything is glued together with systemd so it’s actually usable in a live setting. rtpmidid runs as a service, avahi runs for discovery, mpv is started by a show-player script, and the bridge script sits there waiting for both the midi port and the mpv socket to exist. once the machine boots, the whole stack is just there, idle, waiting for ableton to connect meaning a quick setup is possible without much fucking around.
-
the correct signals to use are midi realtime + song position pointer. ableton sends spp messages that tell you where the transport is, and realtime messages like start/stop/continue that tell you what it’s doing. the trick is that spp by itself is not a play event, it’s just a seek. so what you do is you arm when you see “songpos == 0” (meaning the transport is at the beginning), and then you actually trigger playback when you receive a “start” message while armed. that way you only fire when the sound engineer actually presses play from the beginning, not when they scrub around or restart mid timeline.
on the video side it’s mpv, because mpv is basically perfect for this. you start it once, paused, with a unix ipc socket exposed. something like “mpv --pause --fullscreen --input-ipc-server=/tmp/mpv-show.sock video.mp4”. that socket is just a json command interface. the python script connects to it and sends commands like “set pause false” or “seek 0 absolute”. this is way faster and more reliable than spawning a new player every time, and it removes startup latency almost entirely (~0.1ms latency at most, wow!).
everything is glued together with systemd so it’s actually usable in a live setting. rtpmidid runs as a service, avahi runs for discovery, mpv is started by a show-player script, and the bridge script sits there waiting for both the midi port and the mpv socket to exist. once the machine boots, the whole stack is just there, idle, waiting for ableton to connect meaning a quick setup is possible without much fucking around.
the reason this works and link doesn’t is that link is fundamentally not a transport sync system. link shares tempo and phase relationships, but every participant has its own independent timeline and its own quantum, and start/stop is not guaranteed to be aligned. it’s designed for jam sessions, not deterministic playback.
what i’m doing here is the opposite: ableton is the single source for transport, and everything else is just reacting to explicit events..the nice side effect is that this is fully cross platform without any weird hacks. macos and windows (using the RTPmidi driver) both already speak rtp midi properly, and linux just joins via rtpmidid. the video machine can be completely headless, minimal, and stable, which is honestly a better fit for stage visuals anyway.
you don’t need resolume or touchdesigner or any heavyweight visual stack if your requirements are basically “play this video exactly when i press play”.so all in all, the whole thing ends up being less about “syncing” and more about “stop pretending and just send the right signals for fucks sake”. ableton already knows exactly when playback starts, so just export that over the network and treat it as gospel. everything downstream becomes trivial once you do that.
-
R relay@relay.an.exchange shared this topic
