It has become among the most popular open messaging protocols, but it remains a scalability nightmare.
It's literally the Blockchain of messaging. Its current state is given by the sum of the whole chain of events received since t=0. It's a brittle append-only ledger, and any modifications you do to it are very likely to break it.
Do you have a huge 100GB database and you want to clean up old stuff? Sorry, you can't. There are some non-official solutions for compacting the events, but they're all likely to break your db - and none worked so far in my case.
Do you have users on your instance that entered a busy room on matrix.org? Then you'll get all the join/leave events of that busy room on your db, with no way of deleting them, and nothing to do unless your users exit those rooms.
Did you start your instance by toying with the default SQLite backend, and now that it's become big you want to move to Postgres? Sorry, no official guides provided, only unofficial procedures scattered across a bunch of blogs.
Do you want to change the name of your server? Sorry, you can't. All the events are tightly coupled to your server name. The advised solution is to simply start a new instance.
Such a rigid and brittle implementation shouldn't have become an open de facto standard without much questioning about its poor design decisions.
At the very least, some official tools must be provided to enable admins to compact events. If the size of the database is guaranteed to increase indefinitely, then entry barriers against self-hosting are only going to increase.
Matrix is deliberately designed to NOT require all events since t=0. It requires the key-value events (i.e. state events) for the room, but not the actual messages. In turn, room DAGs can and are split into partitions called 'chunks' whenever there's a gap in federation.
Similarly, as others have said, synapse_port_db for sqlite->postgres should absolutely work (if not, it's a bug. there have been some in the past, but as far as we know it works currently).
@fabio The further #Matrix goes the more obvious it becomes we should've stuck to #XMPP. The developer time wasted on Matrix is immense, imagine if that effort would've been stuck into making a nice XMPP server implementation that plays well with #Docker or #Kubernetes, with all the basic features people desire baked in by default. They would've had time to spare on a nice client if they felt this was needed to make a nice complete package to market.
@tyil@fabio 👋 Hi I stumbled across this thread, but felt like I could slide in a little plug for snikket, maybe?
These folks took prosody and made a really nice package for folks to get started with their own XMPP server (and client offerings) that offer an extremely good out of the box experience. (A rating on the conversations.im auditor too)
Don't think I heard of #Snikket yet, I should check it out. When I set up #XMPP again I want it to run on my Kubernetes cluster, since that's where most things already live anyway. I also want to be able to have bridges to other platforms that I use, such as #Signal and (sadly) #Matrix, so I can have one good client for all my chat again, like we had back in the Ol' Days.
@tyil@chuck@Menel I’ve been running my personal XMPP server for a while on one of my Arch servers. I’ve followed little more than the Arch wiki page for Prosody and done some extra DNS tweaking, everything worked almost out of the box.
The main thing that keeps me invested in my Synapse server is the insane amount of bridges available for Matrix. Actually, bridges are like 90% of my Matrix use-case. I used to use bitlbee to port everything over IRC earlier, but too many plugins started breaking, too many former libpurple integrations have fallen unmaintained/undocumented, and Matrix gave me everything I lost by abandoning bitlbee and much more.
If only I could have WhatsApp, Messenger, Telegram, Signal, LinkedIn, Instagram, IRC, Slack, Google Chat, SMS, etc. all bridged over XMPP, as it is currently possible for me to do with Matrix, and also have only one app installed on my phone for all those services, rather than N (thanks to Element), and also not have to worry of discrepancies between different client+server implementations, I’d definitely drop my fat Synapse server, free up 2GB of RAM and 100GB of disk, maybe replace it with a thin personal Conduit/Dentrite server only to host the Platypush channel, and move everything else to Prosody.
XMPP is the way we were supposed to go from the very beginning, but it went through too many hiccups through in the past 15 years. Eventually, the success of a messaging protocol in these years is done by the number of integrations that you have and the ecosystem that you manage to build around it. That’s IMHO where Matrix won, and too many of us ended up flirting with a technology that is basically Blockchain but for messaging…
@fabio
I don't see many implementations of matrix. There are more server and clients for xmpp then matrix, there is basically only one stable server for matrix, by the company that made the protocol... And matrix can't still claim its just because it is so new... The years flew past.
Now you have the same situation as with every federated protocol, it has to slow down, or you can't have independent, working implementations, or move fast and be just the protocol of one company. @tyil@chuck
@fabio
Meanwhile xmpp wasn't as fast to jump on mobile, but some years later it had new xeps, for mobile, and new clients. And I'm sure, if a micro fraction of the money that was dumped on matrix had found its way to xmpp, that could have been even faster.
Also I doubt matrix bering bigger, it may have more public rooms hosted. Total server and user count is doubtful.
Most of the feeling of matrix beein big is PR money and projects hosting rooms at matrix.org @tyil@chuck
Add comment