The current solution includes blocking Mastodon as a user-agent, which would disable our link previews, making it look like spam and uninformative.I wouldn't say that is the current solution, it is a solution
You can also work to reduce the effects. You know the user agent format, you can provide a reduced page which takes less on your server. It's not a perfect solution but at least improves things a bit.
And I don't know if mastodon was designed expecting to have so many instances running. As we've learned from the Internet, some duct tape solves it
