i suspect the problems with matrix as opposed to signal(signal group chats do work, subject to their scaling factors) is a fair few factors
wasn't designed as an encrypted tool by default, so features aren't gated on "how does this work in an encrypted room?"
linear, consistent history. you will always see every message(assuming the signal servers aren't fucking with you) in a consistent order, there's no disagreement over message ordering, or discovering messages that are backdated.
there is one client implementation and one server implementation. there's no room for "oops a third party client/server made a bug that broke e2ee" since that doesn't exist.