michael, (edited )
@michael@thms.uk avatar

It looks like relay.fedi.buzz is going to break once 4.2 comes out:

https://github.com/mastodon/mastodon/pull/23989

This is VERY bad news for users of small instances like myself.

I do understand the reasoning behind it, and I'm not sure there is a good alternative. But it sure feels a bit like a deja vu from the self hosted email days ...

DataDrivenMD,
@DataDrivenMD@fedified.com avatar

@michael Agreed, this feels like a step backward. I'm curious if @Mastodon and/or @stux would be open to setting up a read-only relay service endpoint from the servers they manage. As 50-60% of posts come from users on their instances, such a move would offset the impact this breaking change

alwynispat,

@michael @stefan oh man.. really liked how fedi.buzz helped to connect with other instances.

moon,
@moon@akiba.social avatar
christian,
@christian@aldr.social avatar

@michael The hashtag relays provided by FediBuzz were a great way to add content I'm interested in into my timelines. I guess as a user of a single-user instance, things are about to get more lonely 😔

michael,
@michael@thms.uk avatar
christian,
@christian@aldr.social avatar

@michael Interesting, I will check that out more later, glad to see there is a GitHub Action option that doesn't touch the infra, as I'm on managed hosting 😅 Thank you 😊

michael,
@michael@thms.uk avatar

@christian yep. That Action was the inspiration for ‘s Action and I’ve run GetMoarFediverse as Action for many months, and can confirm it works well.

anders,
@anders@mastodon.cyborch.com avatar

@michael my impression of @ClearlyClaire is that she is very much anti-small instance. It’s too bad, really.

Mastodon is going to be a significantly more lonely place.

michael,
@michael@thms.uk avatar

@anders @ClearlyClaire my impressions has always been that she’s a kind and helpful person, and she’s personally helped me twice very patiently with hard to debug issues on my single user instance.

anders,
@anders@mastodon.cyborch.com avatar

@michael @ClearlyClaire when I started work on combine.social and tried to have a conversation with her about it, she made it quite clear that they have no interest in making federation better for small instances.

And now we see steps which specifically make federation worse for us.

michael,
@michael@thms.uk avatar

@anders yes. I think mastodon as a whole is certainly not seeing small instances as a positive to put it mildly …

shanie,
@shanie@tails.ch avatar

@michael https://github.com/mastodon/mastodon/pull/23989#issuecomment-1669884175

ClearlyClaire actually responded after all the noise. Not sure if @astro wants to give his two cents.

vyr,

@michael the alternative is to sign the scraper up for an account on each instance it scrapes. they're easy to get on most of the big instances, and they wouldn't generate any more load than any other account that leaves the federated timeline page open all day but never posts.

michael,
@michael@thms.uk avatar

@vyr yes. I suspect that’s what I’d do, if I operated the fedi buzz relays 😁

jdp23,

@vyr Right. That seems better to me: signing up for an account means that TOS apply, and sites that don't want this kind of scraping don't have to approve the accounts. Of course it's a hassle for the scraper especially if they want to go beyond the top N instances, so I can see why there's resistance to it. And it seems like it's taking everybody by surprise so the communication hasn't been great -- these would have been great discussions to talk through before announcing the beta.

@michael

jdp23,

But also @michael I didn't mean to undercut your concerns, when I say "talk through" I mean with small instance admins as well as the fedibuzz folks. The solution as implemented seems likely to favor large instances over small ones, there might be ways to accomplish the goals with a more equitable solution. @vyr

vyr,
vyr,

@michael @jdp23 hilarious update: the thing lets them read your DMs. Mastodon OAuth permissions, while granular, are not that granular.

"donate" tokens only from an account created for scraping purposes, not your personal account, and everything's fine.

https://chaos.social/@defnull/110860106432535258

michael,
@michael@thms.uk avatar

@vyr yeah, I was discussing this with someone yesterday. Seems dodgy as …
When I said I’d get a token for each instance I thought they’d create their own accounts. Not that they’d get random people’s accounts…

@jdp23

vyr,

@jdp23 @michael between this and https://c3d2.social/@astro/110861691765658161 (the relay source is public but not the website that it feeds into) and the fact that https://fedi.buzz/ routinely surfaces Oliphant T0 instances, this project's sketch index is rising fast.

https://github.com/astro/buzzrelay/issues/10 also worth noting this issue Claire flagged yesterday: the relay doesn't forward deletes or updates yet.

michael,
@michael@thms.uk avatar

@vyr the occurrence of sketchy instances doesn’t bother me as much - imho it should be the subscribing instances’ job to block what they want to block.

But deffo feeling sketchy now. Shame, as it was so useful.

I’ll be relying more on going forwards

vyr,

@michael detroit riot city on the front page is still a bad look, like, that's not sketchy, that's a whole racist harassment oil painting

jdp23,

Yeah it's certainly useful but there are multiple "yikes"es here. I'm not sure what the right answer is but this is not looking promising.

Taking a step back, one way to look at this is that the product implementation to date don't address a use case that's critical for small instances and cross-instance communities like Monsterdon.

So a hacky solution emerged -- relying on a security hole, and with some major shortcuts. As a result, small instances and cross-instance communities became reliant on the hacky solution.

Now, the security hole is being closed (good!), but the underlying use case still isn't getting addressed (not good). And, implementers of the hacky solution -- who knew about the upcoming change -- didn't alert the people relying on it, or start exploring an approach that would be less hacky and more secure; neither did the product team. Instead the implementers of the hacky solution waited until the shit hit the fan and then proposed an alternate hacky solution with even worse security implications. Not only that the attention from the discussion highlighted that at least one of the the shortcuts in the hacky solution (not forwarding edits or deletes) that people have been relying on has major privacy and safety implications).

Not good.

More positively, thought, it points out a threat I had missed in the draft Meta privacy threat model!

@michael @vyr

vyr,

@jdp23 this is a general threat model, and Twitter had the same problem. i'd actually expect this from anything with an API:

apps/websites that claim to show you your top posts or who's blocking you or whatever, but are going to be asking for overbroad API tokens or straight up just asking for your password so they can also post spam or access stuff they shouldn't. many of them will probably have shoddy backends and leak credentials or data by accident, or share them with third parties on purpose.

vyr, (edited )

@jdp23 example from last November https://demon.social/@vyr/109814138343110056 (i'll just inline my own post here)

i didn't have "websites that claim to show you top toots and also sell the data" on the bingo board, but i probably should have. anyway, here's "FeedSeer News":

> You grant us read only access to your Mastodon account, and we monitor the feed of status updates from the people you follow, just like your Mastodon client does.

https://news.feedseer.com/about

> We may share aggregate information about our userbase and demographics with investors, business partners, customers, or the public.

https://news.feedseer.com/privacy

vyr, (edited )

for admins who might want to keep an eye on this one particular scraper, as useful as it is, the tokens are created with app name and app website https://fedi.buzz/. (Mastodon autolinking is mangling the URL there, but it's the whole URL from https:// to trailing /).

note that they don't have to be. FediBuzz could just as easily pretend to be Toot! or Tusky or Pinafore. but it is currently politely identifying itself in a normal way.

BentiGorlich,
@BentiGorlich@wehavecookies.social avatar

@michael Hard disagree... I think that relays are a kind of snake oil. It just flushes your instance with a lot of noise, you don't care for. I did try it and it was just bloat for me and the other 10 users on my instance.

michael,
@michael@thms.uk avatar

@BentiGorlich relays: yes. Fedi.buzz: the opposite. Fedi buzz relays are literally the answer to the problem you are describing 😉

BentiGorlich,
@BentiGorlich@wehavecookies.social avatar

@michael Oh yes, you're right, sorry 😅
kinda mixed it up with "the big relay"

daniel,

@michael @astro Just chiming in to say that FediBuzz has been really useful for my single user instance and not being able to use it would be a great loss for me.

michael,
@michael@thms.uk avatar

@daniel @astro check out as alternative

darrendilieto,
@darrendilieto@illo.social avatar

@michael Oh no! That is bad news indeed!

dnddeutsch,
@dnddeutsch@pnpde.social avatar

@michael oh noes! this might be an issue for GetMoarFediverse as well, cc @g3rv4 / @g3rv4

g3rv4,
@g3rv4@g3rv4.com avatar

@dnddeutsch @michael doesn't use the streaming api, so it should keep on working

dnddeutsch,
@dnddeutsch@pnpde.social avatar
lari,
@lari@suomi.social avatar

@michael This is really bad news. Do you @astro have any idea on how relay.fedi.buzz could continue after this?

astro,
@astro@c3d2.social avatar

@lari
I am full of bad ideas :-)
@michael

lari,
@lari@suomi.social avatar

@astro @michael Bad ideas are the best! :)

I've found relay.fedi.buzz really important for my small instance, as none of the largest instances seem to be using any relays. Losing this will mean that my instance will again not be aware of majority of the posts on the Fediverse, and following topics or context of many of the conversations will again be very difficult.

@thisismissem mentions OAuth as a potential solution, but if I understand correctly, this will not be implemented at least in the next version.

thisismissem,
@thisismissem@hachyderm.io avatar

@lari @astro @michael tbh, I don't think the way forwards (long-term) is to use the streaming API to scrape everything that's public, but instead to position the service as valuable enough that instance admins choose to add peerage to your service as a relay.

lari,
@lari@suomi.social avatar

@thisismissem @astro @michael You're probably right about the Streaming API, but I don't think I or any other small instance admin can do much to make the large instances to join a relay. Most of the large ones aren't using any, and I haven't noticed that even those operated by @Gargron would be using them. When Streaming API is blocked, their feeds will be unavailable to my small instance.

This change will probably not have much effect to largest instances, but will affect negatively all the small ones which have been relying on services like fedi.buzz to be aware of what is happening in the

jay,
@jay@toot.zerojay.com avatar

@michael How do other relays work then?

michael,
@michael@thms.uk avatar

@jay through each server admin giving access to each relay they want to participate in. However largish instances don’t usually do this, so fedi.buzz was really useful for small instances to follow relevant hashtags on large instances

KI5SMN,

@michael

Tell us why it will be bad news. Otherwise, it's just neg hype

michael,
@michael@thms.uk avatar

@KI5SMN Because relay.fedi.buzz is essential for small instances (and particularly single user instances) to pull in interesting content.

KI5SMN,

@michael

Thanks for the summary, now Infeel like I know a little about why the change is upsetting some folks.

paul,
@paul@oldfriends.live avatar

@KI5SMN @michael
I use it. It fills my instance with content posts that larger instances, incl Mastodon .Social
have shut us out from by not opening their relays.

"Mastodon and many other ActivityPub-compatible services live and breathe decentralization. Yet it can get lonely on a small instance. Mastoadmins can bring the global buzz of toots into the Federated Timeline of their small server by following ActivityPub relays."
Read more:
https://relay.fedi.buzz/

KI5SMN,

@paul @michael

Is there any hope that this will be addressed by masto devs?

michael,
@michael@thms.uk avatar

@KI5SMN @paul

Don’t think so. I think this is on purpose.

KI5SMN,

@michael @paul

Why would promoters and developers of software, cripple fine grained ?

michael,
@michael@thms.uk avatar

@KI5SMN @paul

Because mastodon is strongly opposed to scraping. And ultimately that’s what this amounts to.

(And because it can be used to circumvent blocks)

paul,
@paul@oldfriends.live avatar

@michael @KI5SMN

Isn't there a mechanism that lets users on a user's basis to stop their posts from being public or does that just set the robots.txt file for the user and people ignore that anyway?

michael,
@michael@thms.uk avatar

@paul @KI5SMN the latter afaik

paul,
@paul@oldfriends.live avatar

@KI5SMN @michael

It was widely discussed in the link he shared. I doubt it.

Last comment was, "Potentially what we could add to streaming is support for:

OAuth: Public. Requires app token + read:statuses if the instance has disabled public preview.

Currently that's not available / not a tested path, as authentication has only been for users in streaming historically."

https://github.com/mastodon/mastodon/pull/23989

  • All
  • Subscribed
  • Moderated
  • Favorites
  • mastodon
  • khanakhh
  • DreamBathrooms
  • ngwrru68w68
  • tacticalgear
  • osvaldo12
  • magazineikmin
  • Youngstown
  • modclub
  • rosin
  • slotface
  • everett
  • kavyap
  • mdbf
  • Durango
  • JUstTest
  • InstantRegret
  • GTA5RPClips
  • tester
  • provamag3
  • thenastyranch
  • cubers
  • cisconetworking
  • ethstaker
  • normalnudes
  • megavids
  • Leos
  • anitta
  • lostlight
  • All magazines