“We rebuilt the Mastodon backend from scratch and made it able to handle Twitter-scale (500M users, 20M writes/second, unbalanced social graph, etc.)”
Congratulations, you reinvented Big Tech.
Did you stop to consider that fediverse (emphasis “diverse”) servers should not be trying to reach “Twitter scale?” That they should be kept small on purpose? That the goal isn’t to recreate Twitter but with some other asshole in charge?
@aral And yes, they are completely missing the point.
One question is if a server that capable means that it’s lighter and cheaper to run for small instances? In that case it could still be somewhat interesting…
All in all it seems that this is just a tech demo for their “Rama” product, whatever that is.
@koen_hufkens It is but they state they rebuilt it from scratch. If so, it doesn’t apply. There isn’t much you can hit them with. These things were always going to happen given that ActivityPub has the same inherent economies of scale that the Big Web has. The only chance the fediverse had of resisting enclosure with its current architecture was social pressure but we’ve all but thrown that away with some of the most prominent folks rolling out the red carpet for Silicon Valley.
@aral TBH, "from scratch" is something I don't buy without proof to the pudding, certainly not in times of LLM and co-pilot. But yes, this is more a philosophical issue rather than just a technical one.
@aral the protocol is open, the rest seems perfectly possible given engineers who have experience scaling web stacks. I wouldn't expect the lay person on the street to know how - but plenty of developers scale this type of thing as part of their day job.
@Paul100@koen_hufkens Indeed. Thank fuck for Facebook/Meta, Twitter, heck ExxonMobil and let’s not forget the Saudi Royal family… where would be without the billionaires and trillion-dollar corporations?
@koen_hufkens@aral If it’s a true clone, then it probably doesn’t share any code, so there’d be no licensing issues. You can’t copyright an API—Oracle tried and failed. In the US, at least, you can’t copyright a database.
@aral But they’re not talking about recreating Twitter. They’re just saying their product is built to scale ‘like a Twitter’, because they know there is going to be a big need for that if govts, nonprofits, institutions and big data providers are to join the fediverse. I don’t see that as a threat, as much as a symptom of our success, and a next level take-off point for the entire fediverse project. But that’s me. YMMV.
@aral do you think killing all mastodon servers would be better than servers that can withstand the threat. I think this is rather stupid. it's a good thing to have other servers and protocols especially if they are more robust.
@aral I never once thought about the fedi (even in the gnu social days) like "I wish this thing would do half a billion users with no issues." That kind of thinking is growth obsession dressed as an attractive engineering challenge.
The more relevant question is, "is it lighter-weight"? I don't care how it scales UP, I want to know if it scaled DOWN. Because vanilla Mastodon, and most of the forks, will pretty much choke small single-board computers or lightweight virtual machines.
@aral I’m sceptical because of the general ‘smell’ and self-congratulation. At the same time Mastodon by all accounts could do with some improvements with efficiency and complexity. I would like to run my own instance, but it really seems like a lot of work to run Mastodon, and need a lot of resources.
I just think it’s not so simple to jump in and dramatically rewrite the code. Mistakes will have been made, things broken.
@aral I mean, reducing the cost of running any mastodon instance by making the backend more efficient generally wouldn’t be a bad thing. But something based on their proprietary tech stack obviously isn’t a good place to start.
@aral Their tech blog post has a revealing sentence:
"It is our belief, however, that such decisions [to limit the size of an individual Mastodon instance] should be product decisions and not forced by technical limitations."
This whole false notion that tech decisions and product decisions can be cleanly separated, is at the root of the SV ideology.
"Technology is just a tool" is the mantra of the tech bro.
@aral@f4grx it looks like its written in Java so its about as scalable as a brick. Tech bros who think a large server farm is OK to run something that should be a lean system with a handful of nodes, instead they build a giant cludgel of trash that’ll gobble up memory.
@aral
"Did you stop to consider that fediverse (emphasis “diverse”) servers should not be trying to reach “Twitter scale?” That they should be kept small on purpose? That the goal isn’t to recreate Twitter but with some other asshole in charge?"
I didn't even read the whole thing but I'm here to assure you that yes, in fact, they did consider that.
Somehow Java is one of the lang's i feel the least for :blobblush:
There is a reason i always go with a 'standalone' or source install instead of containers or docker or whatever, maybe in the future when im old and grumpy :blobcatgiggle:
@aral mastodon is not part of the community, they are a company with a thinly veiled fig leaf of non-profit status (like OpenAI had before their pivot). If we don’t like it, we need to fork it and drive them out.
"First off, we make no comment about whether Mastodon should be scalable. There are good reasons to limit the size of an individual Mastodon instance. It is our belief, however, that such decisions should be product decisions and not forced by technical limitations.
...
Since our instance is just meant to demonstrate Rama and costs money to run, we’re not planning to keep it running for that long."
Add comment