Is decentralised federated social media over engineered?

ryathal, 3 months ago

The fediverse isn’t over engineered, it’s just not quite focused on the right aspects. A federated social network needs to be more like a block chain, where the content is centralized, and the instances (miners) are decentralized. The content is the important part, and with everything being tied to an instance, it makes the content harder to access. You have instances defederating, going down, closing, and version conflicts, all that makes it harder for a network to gain traction.

maegul, 3 months ago

Sounds like you’re describing BlueSky there. Have you looked into it?

Unless you’re talking about something more nostr/web3.0?

sir_reginald, 3 months ago

bluesky depends on one single entity. they promised a lot about their protocol, but they have yet to show that other instances other than the official one can operate in a fully independent manner.

maegul, 3 months ago

I was under the impression that it’s clear that additional relays can work within their system? Have they not setup anything in the protocol for how that’d work?

Ludrol, 3 months ago

But who will moderate the content? Who is to say to what is legal and where? In USA and in EU different pieces of information can be shown. CSAM needs to be removed. Main lemmy devs removed only active mod on !anime!anime due differences in censorship.

ryathal, 3 months ago

The same people that do now?

Ludrol, 3 months ago

Blockchain is immutable so you can’t remove the content.

The same people can’t moderate the content if content is centralised, there would need be an overlord that sets the rules.

ryathal, 3 months ago

Block chain was more on analogy than implementation. The key is that data isn’t bound to an instance, and ideally most people never need to know about instances.

blue_berry, 3 months ago

You are describing a different thing than what the idea of the fediverse is. Content is collected at an instance and these instances federate. That’s why its called Fediverse: people basically form groups, these group federate. It’s a social thing, there is trust involved. With blockchain, the idea is that you don’t need to trust a central entity.

I think you talk about something like nostr.

ryathal, 3 months ago

It doesn’t need to have the full trustless or buring energy for fun, but it does need to be resilient against instances going down, which currently isn’t the case.

Ludrol, 3 months ago

What specific edgecase do you have in mind. The fediverse is coping data and is quite resiliant against data loss.

ryathal, 3 months ago

If an instance shuts down everything from it is just gone. ML already ran into dns issues once, if it goes, 20% of lemmy is just gone.

Ludrol, 3 months ago

It is not gone. You still have an old copy of the post on your instance.

e.g. szmer.info/post/383045

Replaying, posting and making new content is pointless there but the old stuff still stands and it can be read.

blue_berry, 3 months ago

I think thats just a thing that will get better over time

Max_P, 3 months ago

I wouldn’t be on Lemmy if it wasn’t federated.

Why? Because I don’t want to be trapped into another ecosystem that will do a Twitter/Reddit when it gets big enough. Reddit clones are plentiful, but they’re all fragmented in the user base. At least with Lemmy that doesn’t matter, I can use any community from my home instance.

ActivityPub in itself really isn’t that bad, there’s a lot more that goes into the platform as a whole. And it’s also a good choice because it’s a standard protocol that everyone agrees on. Is it perfect? Probably not. But it works good enough, and we now have Lemmy alternatives in the form of Sublinks and K/Mbin. There’s also a decent chunk of toy ActivityPub projects out there as well.

I can still be trapped in a dying ecosystem but that’s true of every site and at least I have the option of taking my data and converting it to another software if I want to.

maegul, 3 months ago

And it’s also a good choice because it’s a standard protocol that everyone agrees on.

Not sure how true this is in practice. See eg: hachyderm.io/ And there’s of course BlueSky and ATProto that may alter that truth drastically (we’ll see)

Otherwise, I hear you. Thing is I’m aiming for something in the middle. If the platforms are all FOSS and commit to easy movement/migration features (which the fediverse sucks at to be fair) as well as good quality aggregators (which the fediverse also sucks at), then maybe your concerns aren’t a fatal possibility?

Also, and I do apoligise for making this a tad personal … it seems you’re on an instance that’s mostly for you or small group of friends (which is awesome! How’s it going?) … which is a level of enjoyment of what the fediverse offers beyond most people and even beyond what many capable of doing so would want to do for their social media. Meeting things in the middle is also about bringing better social media to more people, which could naturally deprive the technically capable of their abilities to be flexible with their needs (though in a way BlueSky’s promise is kinda to do both, if it succeeds).

Kaldo, 4 months ago

I think the biggest issue is account management. Having all these different instances wouldn't be as bad if it were easy to switch accounts or combine their subscriptions, making it truly user-driven instead of depending on the behavior of each individual instance.

density, 3 months ago

There are some tools that can help with this but I don't think completely automatically. I don't use any of them.

See: https://github.com/dbeley/awesome-lemmy#tools - scroll up and down the page too might be something that suits you

maegul, 4 months ago

Exactly. And that’s kinda part of my point. Federation is all about copying content from server to server, so that the central organising structure is the server or instance. Duplicate enough content and a user won’t need to move … is kinda the idea. The reality though is more complex than that, as platforms and their designs get in the way, as does defederation and unpredictable admins. Reality is that the fediverse isn’t user driven, and once you see that, the whole decentralised thing starts to become more questionable (IMO).

So given all of that, I’m wondering what happens if you take away the whole federated idea but still retain some of the aims and principles and try a more straightforward approach that uses the tools and technologies we already have.

sentient_loom, 4 months ago

I still think we need a simple social media protocol that gives me the power to curate my feed rather than hoping my admins don’t defederate with everybody else (followed by hordes of drooling goons telling me to start my own instance).

maegul, 4 months ago

Well that’s kinda the point of my quick suggestion in the original post.

Instead of committing to federation, how about committing to aggregating clients that allow you to do exactly this. Right now, there’s no app that will work for both lemmy/kbin and mastodon/microblogging. No way to unify the notifications or even combine the feeds or just have a unified interface for the two platforms (that are, let’s face, both just full of text messages and feeds).

By allowing each platform to be distinct but remain open with their APIs and “play nice with each other” while leaning into the value of aggregators as a primary part of the value proposition of the system, users might be better served.

sentient_loom, 4 months ago

I have a hard time imagining what that looks like, which is just a failure of my ability to think about these technologies. But what I’m talking about is a little different, simply because I don’t think we can go from these diverse systems into something simple and elegantly connected.

I mean something like email but structured differently. Though email still has spam filters and blacklists, and a new social media protocol might still need those (inevitably infringing on my curatorial freedom similarly to defederation).

My point is that I’m still looking for something new, rather than to reform the defediverse.

Edit:

I might be wrong. It might be good to leverage what we started here and reform the tech to give users more freedom, and take pressure from admins.

Also… maybe email is not the example I should follow. Maybe it’s more like torrents. P2P social media.

maegul, 4 months ago

Well I’m spitballing here, so I wouldn’t worry about not being able to imagine it! I’m struggling too!!

Is there a chance that BlueSky is more like what you’re after?

sentient_loom, 4 months ago

No, that’s fairly centralized too. I think I want a peer-to-peer social media protocol. Maybe more like torrents than email.

maegul, 3 months ago

Yea right. Me too I think. It’s out there and has been for a while. Just don’t think it’s ever taken off.

You might find this interesting: pfrazee.com/blog/why-not-p2p

sentient_loom, 3 months ago

I’m checking this out!

maegul, 3 months ago

Yea I didn’t know before seeing that that one of the BlueSky devs (the author of the blog post) was heavily involved in p2p stuff (eg beaker browser).

melmi, 3 months ago

The fundamental problem is that all this data needs to be hosted somewhere. P2P systems have the issue of persistence: either posts only stick around as long as the people who posted them keep their server online, which is then a burden on anyone who wants to be active in the community, or everyone shares the responsibility for hosting, and then what happens if someone posts CP? Is it just mirrored across the entire P2P system, and each person has to individually root out the CP or just be okay with hosting CP?

Torrents work because you have to actively join a torrent. But discoverability is handled from the outside, through trackers. Trackers choose what they want to host.

Tor or really I2P are the closest equivalents, but they work because everything is encrypted going through them. It’s a privacy thing. With social media, everything is public by design.

sentient_loom, 3 months ago

Persistence could be traded off for decentralization. Just like torrents’ associated data are stored on people’s computers (and the data dies if nobody is seeding) this kind of social media doesn’t have to be permanently stored on a server.

Blaze, 4 months ago

You are on SJW, are you really concerned about defederation? You guys seem to be doing well over there

sentient_loom, 4 months ago

They’re not the worst, but I want literally nobody restricting my access to literally anybody (criminal behavior a grudging exception, and even that I’d prefer to take care of myself).

Also, I fled a couple other places first.

I just don’t want a Mommy and Daddy telling me which servers are Naughty or Nice. I don’t want technology that enables those restrictions at all.

solrize, 4 months ago

Usenet?

maegul, 4 months ago

I do not know enough about how Usenet worked apart from picking up the impression that much of what was done there is being reinvented in the fedi (however accurate that is).

For me, the commitment to having good aggregating and unifying clients, and the commitment to open APIs that would necessitate, is pretty central to my suggestion. Not sure how much of that was in the usenet system (though probably more than I’m aware!).

solrize, 4 months ago (edited 4 months ago)

Usenet servers had no web UI since the web was not yet invented. They ONLY provided an API and the presentation layer was done completely by the client. There were dozens of client programs people used, maybe 100s. Clients handled all features like subscriptions, tracking read articles, boosting or blocking topics or users, etc. So in that regard it was much more private for users. The servers were computers 1000x slower than a modern mobile phone. So yes, Lemmy seems overbuilt.

maegul, 4 months ago

The servers were computers 1000x slower than a modern mobile phone. So yes, Lemmy seems overbuilt.

Well yea, part of my thinking here is whether all the work of federation, from building the software, debugging and testing it (which AFAICT is a huge pain in the ass) and then actually running it as a job … is actually worth it … when users could very well be happy with something much simpler and the mission of creating a more open, safe and “billionaire proof” social web easier to achieve with something more straightforward.

solrize, 4 months ago

Usenet had a counterpart to federation in that posts got automatically propagated between servers. I haven’t read the Wikipedia article about it but that might be a good place to start if you want to learn about it. Being a server admin did require some ongoing effort, just like with Lemmy.

9point6, 4 months ago

The more simple approaches have already been tried and tend to die before they live.

Social media requires a network effect in order to be successful. Given the established players have had nearly 2 decades to accumulate vast networks, it would be a huge uphill struggle to start from zero content and users. Federated & decentralised social media is the answer to this—you get a network for free, giving the software a chance to stand on its own merits.

For this to all work correctly, they must all talk the same, ideally standard, language (the activitypub protocol) and for decentralised software to actually be decentralised, there can be no single point of failure (therefore caching). As someone mentioned, SSO is inherently centralised, even with something like OpenID, if your authority is down, your account is unusable, so it wouldn’t really add much to the experience as it stands (and possibly may risk complicating it more for new users).

maegul, 4 months ago

Yea, buy in and network effects are certainly the tricky part. But that’s also true of the fediverse … it’s been going a long time and in many ways was really “gifted” with Musk’s twitter purchase (seriously, if someone else were to take charge there and reset it back to pre-musk, you’d see a bunch of people leave masto) … and Spez’s API pricing. Before these events, the fedi was pretty quiet compared to now. Lemmy, before the reddit migration was very quiet and may very well have failed by now or soon were it not for the migration.

Moreover, ActivityPub doesn’t get you seamless network effects. Lemmy and mastodon mostly don’t have cross-traffic, and that’s because their platforms basically lack any mutual support for each other. If they worked well with each other, Lemmy would be a much busier place (and masto would be better structured). Same probably goes to some extent for things like Peertube and bookwyrm. There’s also the lowest common denominator effect when it comes to features. One platform may support/provide “Quote posts”. But because Mastodon doesn’t, and they have the bigger user base, it doesn’t really matter, as no one else will see the quote posts and so the new platform doesn’t really have much to offer new users, which in turn basically turns the fediverse into the mastoverse (which is actually happening) and undermines the promise of enabling new platforms with built in network effects. Mastodon could just become one big single server or platform today and many probably wouldn’t mind.

Otherwise, RE SSO, I had in mind that trusted platforms would be mutual sources of authentication such that an account on one is effectively an account on all of them.

squaresinger, 4 months ago

E-mail. E-mail does support small servers.

Btw, I think you are mixing up a few topics here, so let’s see what you actually want.

• Protocols are what computers use to communicate with each other. No protocols means no interaction between different computers/servers. Without protocols, none of the things you ask for can be possible.
• Federated services don’t have single sign on. On the contrary, single sign on is a centralized service not a distributed one. To clarify that: I cannot log into lemmy.world with my feddit.de accout, same as I cannot log into hotmail with my gmail account. In both cases I log into my instance/provider and this allows me to communicate with people on other instances/providers. Federation is the process of sharing content between instances. SSO on the other hand is a centralized service that then communicates with other services to let you log into these other services. For example, I can log into my Google account and then use this to login to other sites. This only works because people trust Google. This would not work as a decentralized service with untrusted servers.
• Duplication is used on federated services for a few reasons. First, it’s a kind of caching mechanism distributing the load. If someone posts something on one instance, it’s transferred only once to the other instances which then serve it to all their users. Without duplication, each individual view would have to be requested again from the original instance. The other advantage is that the admins of all the instances retain control over the content. If the other instance goes offline, users can still see “their” copy of the content. And if the other instance doesn’t moderate their content, the mods/admins of your instance can do that themselves.

So as you see, these concepts aren’t there just for fun, but for a purpose.

maegul, 4 months ago

Protocols are what computers use to communicate with each other. No protocols means no interaction between different computers/servers. Without protocols, none of the things you ask for can be possible.

By “Protocol” I was referring specifically to ActivityPub and the difficulties of engineering against that standard and then the compute resources required to perform the federation tasks. And in addition, the work of inventing a new protocol and the systems around it, as oppose to relying on the protocols and systems that already exist, which I presume would be mostly sufficient to run what I suggest. Obviously at a general level a “protocol” is required for any interoperation between platforms or servers over the internet, so I’m honestly not sure where you’re coming from with this feedback.

---

Federated services don’t have single sign on. On the contrary, single sign on is a centralized service …

I’m not talking about the fediverse currently having SSO.

I was brief in the top post, in part to start a conversation but because it was originating from mastodon. But the “platforms … dedicated to …” part of my post was referring to the idea that a system of trust would be committed to amongst the suggested platforms where they’d be mutually trusted sources of authentication.

---

Duplication is used on federated services for a few reasons …

I said “over engineered” not “just for fun”.

The premise of my post is to question and ponder the tradeoffs involved. Keeping in mind that social media has for a long time and continues to operate through centralised servers with many many people being very happy with it, the UX and development issues created by the underlying design of the fediverse may not be worth the technical features.

Implicit in my post is the idea that decentralised/federated social media might be a bit of technological idea that technologists like without thinking through the practicalities and usability issues. Many have bounced off of the fediverse for essentially this reason … because in the end the point of social media is the social part not the independent decentralised data resilience part.

I like the fediverse as much as anyone … I was here before the reddit migration promoting lemmy and kbin on mastodon. But I’d prefer an open and free social media space that works for people rather than one that’s futilely married to a tech idea and particular implementation (ActivityPub).

In the end, I’m wondering if there’s a middle way and have simply put up my first attempt at a set of ideas.

---

Otherwise, beyond the substance of your reply, I have to say that your reply was unnecessarily condescending. My post was rather short, so thinking it was without any meaning or coming from a completely ignorant place makes some sense, but without really having much to offer to the actual issues or discussion in your reply (you seem to have missed the essential thrust of simply having a few large servers committed to being open and working together as an easier alternative to big-corp social media), you managed to also make it clear that you thought my ideas were silly or juvenile. Perhaps I am being silly, but I feel your tone and presumptions were unnecessary.

Blaze, 4 months ago

as oppose to relying on the protocols and systems that already exist,

Which alternative to ActivityPub would you suggest?

maegul, 3 months ago

Well, none really, because the work of ActivityPub wouldn’t be performed in my idea. There wouldn’t be mass copying of content to all servers. And the trade off would be that you wouldn’t be able to follow or see content from another server either (just like modern Twitter and Reddit).

The work around though is that SSO should enable easy cross posting from one server to another, so information/posts would be easily shared between servers. But more importantly would be the commitment to open APIs and building/supporting aggregating or unifying clients, so that from a single interface you can follow and read and reply to content from any server.

In terms of servers and users, the idea is essentially to do the content copying/unifying at the user side rather than the server side.

As far as what protocols are necessary here … basically the internet we have now … which is very much the point.

In simple terms the idea starts from a common praise of the fediverse … that it enables posting between Twitter and Reddit and that it is open. Well, how about we build Reddit and Twitter clones just with FOSS, open APIs and cross posting features?

Blaze, 3 months ago

In simple terms the idea starts from a common praise of the fediverse … that it enables posting between Twitter and Reddit and that it is open. Well, how about we build Reddit and Twitter clones just with FOSS, open APIs and cross posting features?

Isn’t that opening the door for future enshittification? Like as soon as Lemmy/Mastodon reach 100k users, a big corporation buys the only instances (as there is no federation in this scenario) to the admins, and turn it into a corporate product? What’s preventing that without federation?

maegul, 3 months ago

Good point!

I’d say more or less the same thing at the core of the fediverse, which is an open and shared commitment to values, features and practices. If the platforms are FOSS, and you can always download your content/posts, than any shitification will be met with moving, no?

There’s the risk of losing all the threads attached to the posts, I suppose. But I’m not sure that that’s an absolute problem. There could be solutions around commitment to archiving and hosting archives etc. But also, the federation thing of copying everything everywhere creates problems around privacy and safety (like people have left the fediverse because they’ve found it to be worse in terms of racism etc than twitter). Having content on single servers with higher walls allows for better safety and privacy features. So it’s a trade off.

As for a big instance being bought … well like I said, people can just move. Shared sign on should make that a nicer experience than here. And some feature for hosting archives of posts from other servers (which the structure of the fediverse actively makes very difficult because content has to be uniquely addressable across the whole network) could probably go a long way to protecting good content. On the fediverse, people have to move whenever a server goes down, but can’t take their posts with them.

On top of all of that I’m not entirely clear on how much a federated protocol prevents corporate enshitification. When Threads federates, and should for example something like Tumblr and maybe some others do so such that the majoriyt of the fediverse is actually on corporate platforms funded by ads and data tracking (which can apply just as well to content that federates with such platfroms from otherwise FOSS platforms) … is that not some form of enshitification? Which isn’t to touch on the things that can happen to the ecosystem once big-corp wield such large user bases.

What protects the fediverse then? I’d guess it’s that the platforms we have are FOSS and those of us who want to break away and start our own instances can. But that’s not the protocol or federation … that’s FOSS platforms and commitment to values and working together. If I’ve got a point with this … the broader abstract idea is that many of these problems are not tech problems but people problems, and, very much to my broader point, confusing people problems with tech problems can lead you astray into making solutions that do not succeed because you’re blindsided by your tunnel vision love for the tech solution.

I’m genuinely curious to know if/how I’m wrong about this. Because a decent read of the fediverse in the wake of the “2023 migrations” is that, to many people, federation itself was form of social media shitification … in that it was seens as an overly complex and annoying technology that actively disrupted and sometimes worsened the core features and motivations of social media … socialising with people … all while acolytes could only tell complainants that their issues weren’t meaningful or serious enough and were likely due to them not using the fediverse correctly. Now I’m here, obviously, because I’m a fan. But I can’t shake the possibility that the fediverse might have fatal flaws and am always a fan of solutions that stick with what’s easy and tried and true (generally good engineering advice IMO).

squaresinger, 3 months ago

Sorry, no condescension intended.

Your post read like one written by someone with very minimal knowledge about the subject, which might have been a misunderstanding on my part. So I tried to cover the basics before talking about the rest.

There is really no shame in asking questions about something where you don’t have experience. There are far more topics I have no idea about than there are topics where I do have a deep understanding.

So to get on the same page, I’ll summarize what I understood, please correct me if you mean something different.

• You don’t like ActivityPub, you want a new protocol
• The system should make it easy to create new, small instances
• The instances should share sessions with the other instances (=single sign on) based on trusting them
• You prefer a centralized system?
• You want the system to not use a single protocol (ActivityPub), but use multiple protocols?
• ActivityPub based services have bad UX due to the complexity of the protocol

Is this correct?

We have a few contradictions here.

You cannot have a system where anyone can easily create servers and at the same time have shared sessions based on trust. These two requirements conflict with each other.

Either servers only work with servers they trust, and then you can’t just create a new small server and interact with the network.

Or anyone can easily create a new small server, but then you can’t do anything based on trust, since you never know if that server was created with malicious intent.

Regarding centralized/decentralized you have to differentiate between implementation and management.

All major social networks run distributed systems. If you want to serve billions of users, you need to run millions of servers. These servers are distributed around the globe to give fast access to users everywhere. Chances are pretty high that your ISP has a few racks of Facebook, Netflix, YouTube and Tiktok servers.

Their distributed system is orders of magnitude more complex than everything running ActivityPub combined.

But their system works, because they have tens of thousands of highly paid specialists to make them work.

ActivityPub based services on the other hand have almost no funding and manpower.

Mastodon is the best in this respect. They have 6 people who are actually working on the system.

Lemmy has two developers who earn close to minimum wages.

Kbin has a single guy developing it.

That’s the real reason why the UX is crap.

If anything, ActivityPub and the services running on them are extremely underengineered and underdeveloped.

Btw, there is something rather close to what you seem to want: online forums with Google single sign on.

The forums are not interacting at all with other forums. No federation or anything at all. There are enough commercial solutions that work really well. And with Google Single Sign On you also don’t have to register for each forum.

maegul, 3 months ago

Sorry, no condescension intended. …

All good!!

And thanks for the reply! I don’t have time now to read through it, but will later. Just wanted to say I’m aware my post was very short and understand where you were coming from … and that, for me, if I address an issue I have with tone, it’s in the interests of the community, kinda “tone policing” as awful a term that is to say out loud … to just let someone know they may not come off as well as they or we would like. But I’m all good personally and look forward to reading your post!

smileyhead, 4 months ago

HTTP is all you need 👍