Missing search results when looking for federated communities (at Lemmy)

Prouvaire, 5 months ago

This is one of the biggest issues and barriers to discoverability with the Fediverse in my opinion.

As I understand it, unless an instance has already subscribed to a community (magazine in kbin parlance), then in order to make that community (magazine) appear in your own instance, you need to:

• First search for the community (including the community's home instance) name in the magazine search function.
• The search will come up blank, but the act of searching for it will trigger a backend request for your instance to start federating content from that community. However there's no message to tell you that it's doing that. It just looks like that community doesn't exist.
• Further, it may take up to several days (in my experience) for federation to start, ie, you have to repeat the search for the community and only then can you subscribe to (follow) that community
• And when it does start, it only starts grabbing new content. So first it looks like the community doesn't exist, then it takes a long time for content to appear, and then it looks like the community is sparsely populated unless you go back to the community's home instance, rather than staying in your own instance, to catch up on old content.
• Further pinned posts aren't federated (at least between lemmy and kbin I believe), so you can't even rely on a "here's what you need to know" introductory post to orient new members.

Contrast this to reddit, where (because it's a centralised system) searching for a subreddit produces immediate results, you can join a subreddit immediately, and you can immediately see all current and past content for that subreddit. Much more intuitive and useful to users.

Unfortunately the activitypub protocol that underlies lemmy and kbin doesn't appear to have been designed for reddit-like communities in mind. Ie communities that tend to feature long-form content, posted relatively sporadically, and where having access to the community's archive is very useful to members. It works somewhat better for twitter-like communities where it's easier to jump in "mid-stream" and - because posts tend to be only a few words long - you're more likely to start seeing new content after only a short delay.

I wish that this is something that's addressed at the Fediverse level.

Doll_Tow_Jet-ski, 5 months ago

Thanks, that's valuable information to know

rhythmisaprancer, 5 months ago

Thank you for posting this! A nice refresher as I had forgotten.

kopper, 5 months ago

there actually is a way to "backfill" content, being the outbox. (although it was not meant to be used this way, from what i can see from reading what little documentation is out there)

lemmy uses it to federate about 10 or so (i think it was?) of the latest posts, and most microblog implementations use to federate the "pinned" posts of an account

activitypub has quite a lot of quirks, both spec-wise and implementation wise, and there are many reasons including an apparent near civil war inside the working group which resulted in this messy state it's ended up in

honestly the fact that this protocol works at all is a miracle

Doll_Tow_Jet-ski, 5 months ago

I'm a bit lost in the jargon. What is backfill and what is the outbox?

kopper, 5 months ago

@Doll_Tow_Jet@kbin.social I don't know if that's the official word for it but I'm using "backfilling" to refer to loading the history of a person or group after first federating with it.

the outbox is a special collection (list in activitypub speak) that's intended to work kinda like an old school email outbox where you put messages to be delivered and it would deliver them to the inboxes of people (and servers that were offline at that time would later pull them in from the outbox)

or, well, that's what the spec says. nobody uses it like that in reality because activity pub was intended for a completely different kind of social media than how it ended up being used (it seems to expect more "everything apps", including an entire client api that would completely abstract away the instance into nothing more than a dumb pipe for activities unlike the current reality where instance software dictate what you can do)

this probably confused you even more but it's getting pretty late here so I can't words good, sorry!

ghostatnoon, 5 months ago

It's usually easier to search via external sites (https://lemmyverse.net/ and https://browse.feddit.de/ are the ones I've been using). You'll still have to manually copy over the magazine name in order to subscribe for it, but you'll get a wider variety of results without having to go to each instance individually.

Doll_Tow_Jet-ski, 5 months ago

Thanks for the tip, that's a good workaround

Crul, 5 months ago

You can access communities through the URL:

https://kbin.social/m/COMMUNITY@INSTANCE

Example:

kbin.social/m/cassettefuturism@lemm.ee

Doll_Tow_Jet-ski, 5 months ago

Thanks, that worked, although I cannot see any of the posts in the original community :(

Crul, 5 months ago

I cannot see any of the posts in the original community :(

I cannot helps with that :/

I know federation is not perfect. I would suggest trying again later just in case it’s taking some time to get the data from the other instance.

Doll_Tow_Jet-ski, 5 months ago

Thanks, I will