Question about how communities are discovered from other instances

I have a lemmy account on lemmy.sdf.org

When searching for communities to join I can see several in the list from lemmy.ml and beehaw.org

but while searching for some communities that I know exist on lemmy.ml or beehaw.org I am getting no results

will a community from instance A only appear in search results on instance B if a user from instance B has already manually subscribed to that specific community from instance A?

basically, there is no scraping of 'all available' communities even when two instances know of each others existence? only 'specific communities' are indexed when a user has manually found/subscribed?

kiwi,

If nobody from your instance is already subscribed, then you can only search for the community by the full url. I wrote a bit more about this here: https://lemmy.directory/post/34207

lackthought,
@lackthought@lemmy.sdf.org avatar

that lemmy.directory looks great!

a minor issue with this workaround is that when I search for a community they all show 1 subscriber (your scraper obviously) and no more information about the true size of the community, you have to click on them to see which ones may or may not have user activity

so if the original design of lemmy is to only index previously subscribed communities it will be difficult for smaller/niche communities to grow and become discovered (unless someone uses your directory obviously)

hopefully the standard lemmy search can be changed so it is more universal by default and with better metrics rather than just # of subscribers from the current instance

kiwi,

Thanks for the feedback! I think if your main goal is to just search for communities by name, then it’s best to use: https://browse.feddit.de/.

I find lemmy.directory the most helpful in browsing all posts across the lemmyverse in a single feed (via the “All” filter).

Searching for new communities may be able to be made more universal, but automatically pulling in those feeds (subscribing) is probably not a good general idea. See here for more info on that: https://lemmy.ml/comment/476925

lackthought,
@lackthought@lemmy.sdf.org avatar

how does your directory (or the search from feddit.de for that matter) find brand new communities?

suppose I create a community on sdf.org called 'HooverMaxExtractPressureProModel60' for discussion on a specific vacuum cleaner

how will any 3rd party scraper/search tool know of its existence so it can be indexed?

I was thinking each instance could keep of a list of all their own communities and share that list with other known instances in some way as a way to 'spread the word' on what is available

kiwi,

I think that would be possible to do to help with searching for new communities. Although for now I’m not sure there’s too much difference between it existing on each instance vs on https://browse.feddit.de/. Other than ease of use of staying on one website I guess.

lackthought,
@lackthought@lemmy.sdf.org avatar

yeah for me its mainly laziness so I don't have to use an external site to search

but new users who may not be aware of external directories could have trouble finding niche communities if no one from their instance has subscribed yet

just a user experience thing I was thinking about

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • lemmy@lemmy.ml
  • Durango
  • DreamBathrooms
  • khanakhh
  • osvaldo12
  • ngwrru68w68
  • magazineikmin
  • InstantRegret
  • thenastyranch
  • rosin
  • Youngstown
  • slotface
  • cisconetworking
  • kavyap
  • tacticalgear
  • JUstTest
  • GTA5RPClips
  • cubers
  • mdbf
  • ethstaker
  • everett
  • modclub
  • normalnudes
  • provamag3
  • tester
  • anitta
  • Leos
  • megavids
  • lostlight
  • All magazines
    •