Has anyone defederated your instance? An update to the Defederation Investigator

A few months ago I released the Defederation Investigator, a tool to verify the federation status of Lemmy instances. With this new update, I’ve expanded it to support multiple Fediverse softwares, including:

  • Mastodon
  • Misskey
  • Mbin
  • Pleroma & Akkoma
  • Friendica

https://lemmy.basedcount.com/pictrs/image/dd1d4e1d-59ac-426c-95a2-439326d6a005.png

This works both ways: you can verify which Mastodon (et al) instances have defederated your Lemmy instance, as well as check the federation status of an instance running any of the supported softwares.

Like most of my works, this tool is FOSS and available on my team’s GitHub.

Limitations

Many microblogging platforms, Mastodon included, offer admins the possibility of hiding their blocklists from the public. As it turns out many instances have chosen this approach, so the available information can be pretty limited at times.

Also, this update has increased the pool of instances from a couple hundred to over 2 thousand, so query times have increased significantly. You can reduce them by deselecting some softwares from the query page (hint: most fedi instances are Mastodon ones, so by deselcting them you’ll cut out over half of the pool).

What about Kbin?

To my knowledge, Kbin doesn’t share its federation status through an API like most softwares do. Furthermore, given recent events, I have little faith in the Kbin project. Instead, I chose to support its community driven fork: Mbin.

What about Peertube and Pixelfed?

I tried looking through their API docs and wasn’t able to find any endpoints sharing either federation or defederation statuses. If anyone is familiar with any of these softwares and has any ideas on what to do to retrieve such information feel free to contact me, I’d love to add support for both.

What about …?

Want more softwares? Feel free to propose them. I’d like for this tool to support as many projects as possible.

tekakutli,
@tekakutli@lemmy.dbzer0.com avatar

based

gabe,

Oh my God. This is really, really, REALLY bad. I know this looks like a useful tool but extending this to software that isn’t lemmy is going to make the rest of the fediverse enraged. Tools like these are used as active harassment vectors. Theres a reason why the other fediverse instances that make these tools are affiliated with kiwifarms. Seriously please consider disabling non-lemmy software interaction immediately!

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

I’m sorry could you please elaborate on why the rest of the Fediverse would be enraged, or how this could be used for harassment? I don’t think I follow. I’ll admit, I only interact with the Fediverse through Lemmy so maybe there’s some dynamics of the Masto-sphere I’m not picking up.

My understanding is that Mastodon admins can choose to hide their /domain_blocks endpoint to either outside users or even to all non admins. (source), and as a matter of fact almost a thousand of the 1700 Mastodon instances I’m querying already do so, so really I can only get the federation status of the few hundred that remain.

I think the admins that prefer not to show their defeds, in fear of harassment, are already hiding them, so it should be ok for me to query the remaining ones.

gabe,

There are aspects of the wider fediverse culture that is ignored on lemmy, and thankfully its lack of interoperability has helped it avoid trouble (for now). I’m really not sure how to explain to you how it could be used for harassment as it requires a bit of detailed background to how the wider fediverse functions and its history and I don’t really know if it would convince you if I did explain it in enough detail. Just because data is publicly scrape-able doesn’t mean it’s acceptable to do so.

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Well of course I can’t guarantee that I would be convinced, even after hearing that but explanation aside

Just because data is publicly scrape-able doesn’t mean it’s acceptable to do so.

Isn’t it? If, an instance admin, has the possibility of hiding some data to the public and refuses to do so, it’s either:

  1. Because they are fine with the public accessing it
  2. Because they are ignorant and unaware of such a feature, which I honestly don’t think is an acceptable excuse (after all users have entrusted this person with their data, ffs)

At the end of the day what I am doing is nothing more than what any user could do by checking the “Moderated servers” section of the about page of any Mastodon instance.

I’m sorry but I’m really am not seeing the logic behind your point.

gabe,

Do you know what kiwifarms is? Just curious.

Nerd02, (edited )
@Nerd02@lemmy.basedcount.com avatar

I didn’t, up until yesterday night when you mentioned it. Had a quick Google search and read the wikipedia page, holy fuck there’s some sick people out there. But I still fail to see how defed.xyz could help them doxx or otherwise harass people.

I don’t want to be the author of software used for harassment, obviously, but I don’t think you could use my tool for that, even if you wanted to.

mulcahey,

Thank you for this great tool!

Is there a way I could use this to find a Lemmy instance that let’s me interact with desired instances? For example:

  • I want to participate in Instance A, B, and C.
  • A has defederated from B and C
  • can I generate a list of instances that let’s me interact with A, B, and C? Then I’ll move my account there.

Thank you!

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Yeah sure. Assuming you are only targeting Lemmy instances (other softwares make this a bit more complicated), A “can interact” with B if:

  • A hasn’t blocked B
  • B hasn’t blocked A
  • Neither A nor B are on allowlist. If either is on allowlist, it must have explicitly added the other one to its allow list (this is very uncommon, the only big instance using allow lists is hexbear.net)

So, to verify this, you could query the Defed Investigator with the instances you care about, one at a time. Only select the softwares you care about (likely only Lemmy) to make the query faster. Say you wanted to verify the compatibility between lemmy.world and sh.itjust.works (just making an example). Go to defed.xyz/check?name=sh.itjust.works&software=lem…

  • ✔ lemmy.world doesn’t appear in the “Instances defederated from sh.itjust.works” (this means .world hasn’t blocked SJW)
  • ✔ lemmy.world doesn’t appear in the “Instances defederated by sh.itjust.works” (this means SJW hasn’t blocked .world)
  • ✔ lemmy.world doesn’t appear among the “Instances not allowing sh.itjust.works” (this means .world isn’t on allowlist or, if it is, it has explicitly allowed SJW. Again, this is very uncommon)

Also make sure the instance you are looking for isn’t among the “Instances that returned errors”, of course.

captainastronaut,
@captainastronaut@seattlelunarsociety.org avatar

Neat thanks!

kurumin,
@kurumin@linux.community avatar

Hi is the automod and user flair projects still active?

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Hi. Yeah, both are still active, up and running.

User flairs are currently invisible on my instance because I didn’t get the chance to update my modded UI to the most recent Lemmy version, but the microservice itself is still perfectly operational.

kurumin,
@kurumin@linux.community avatar

Pretty cool man! Have you considered making a PR to include the flairs to lemmy repo?

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

I have, even Nutomic asked me to, but the thing is I’m a bit of a noob when it comes to Rust. That was my first Rust project ever and the Lemmy backend is big and scary. I would rather gain some proficiency with the language first. Plus, it would likely have to be structured somewhat differently than my implementation if it was integrated into Lemmy proper.

kurumin,
@kurumin@linux.community avatar

Oh I get it, I am in the same stage of Rust myself. I want to contribute in the future though.

UndercoverUlrikHD,

0 defederated, 1 federated (lemmy.world) and 2091 errors?

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Not quite, lol.

https://lemmy.basedcount.com/pictrs/image/ab80e170-5bd7-4668-82e2-0115c5bf90bf.png

https://lemmy.basedcount.com/pictrs/image/bc06953d-7e0d-4dcd-9768-9009732d9599.png

https://lemmy.basedcount.com/pictrs/image/ef2f852c-21dd-4d81-9210-24ce6b7322e1.png

Some errors will happen, that’s inevitable, CORS is a bitch, but if you are getting that many I’m going to make an educated guess and assume there’s something wrong with either your device or your connection.

UndercoverUlrikHD,

Tested my local instance, lemmy.world was the only one that returned as federated to programming.dev

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Ah so you were querying programming.dev? In that case I got:
0 defederated from; 3 not allowing; 111 defederated by and 462 federated

Where are you running it from? Because… it works on my device.

UndercoverUlrikHD,

Apologies, it was a mistake on my end. Tried it again and noticed uBlock going wild. It was blocking every website except for lemmy.world which was globally whitelisted in my settings.

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Uh interesting. Did you have any fancy custom configuration? Maybe you were blocking all requests to other hosts or something like that? I also have ublock origin and it didn’t give me any troubles.

UndercoverUlrikHD,

Nothing fancy, I simply block all third party content unless it’s whitelisted.

Oha,
@Oha@lemmy.ohaa.xyz avatar

Wouldnt it be smarter to query the blocked list on the backend like once a day instead of doing thousands of requests in users browsers?

iso,
@iso@lemy.lol avatar

Well, that’s probably requires a back-end server and a database.

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Like @iso said, can’t query from the backend if there is no backend :)

It’s all serverless, so this is as good as it gets.

Blaze,
@Blaze@discuss.tchncs.de avatar

Nice

originalucifer,
@originalucifer@moist.catsweat.com avatar

i de-federated the defunct artemis.camp

does not show up on the list for my mbin instance https://moist.catsweat.com

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

There seems to be some problem with your instance’s API. While the federation page works fine, when polling the defederation API at

moist.catsweat.com/api/defederated

I am getting the following error:


<span style="color:#323232;">{
</span><span style="color:#323232;">    </span><span style="font-weight:bold;color:#183691;">"type"</span><span style="color:#323232;">: "https://tools.ietf.org/html/rfc2616#section-10",
</span><span style="color:#323232;">    </span><span style="font-weight:bold;color:#183691;">"title"</span><span style="color:#323232;">: "An error occurred",
</span><span style="color:#323232;">    </span><span style="font-weight:bold;color:#183691;">"status"</span><span style="color:#323232;">: </span><span style="color:#0086b3;">500</span><span style="color:#323232;">,
</span><span style="color:#323232;">    </span><span style="font-weight:bold;color:#183691;">"detail"</span><span style="color:#323232;">: "Internal Server Error"
</span><span style="color:#323232;">}
</span>

this also gets interpreted as an error by my tool, which is why the green “Instances defederated by …” sections is empty.

originalucifer,
@originalucifer@moist.catsweat.com avatar

i am running the standard mbin, no mods. i suspect youd have the same issue with all mbin servers

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Actually I don’t. After having tested it on the biggest mbin instances, it worked fine on:

  • fedia.io
  • kbin.run
  • kbin.melroy.org
  • thebrainbin.org
  • rimworld.gallery
  • gehirneimer.de
  • mbin.0ut0f.space

However I did get the error on:

  • kbin.chat
  • kbin.life

So whatever it is, it’s not just you. I’m afraid I can’t investigate any further without looking at some instance logs, though.

originalucifer,
@originalucifer@moist.catsweat.com avatar

interesting! now i need to know

originalucifer,
@originalucifer@moist.catsweat.com avatar

Fixed! my instance api was broken due to a pathing issue which is resolved, and your tool now functions as expected. thank you!

Nerd02,
@Nerd02@lemmy.basedcount.com avatar

Nice! Glad you were able to find and fix an issue with your instance.

Annoyed_Crabby,

Nice tool!

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • fediverse@lemmy.world
  • rosin
  • DreamBathrooms
  • everett
  • magazineikmin
  • InstantRegret
  • ngwrru68w68
  • Youngstown
  • mdbf
  • slotface
  • vwfavf
  • tacticalgear
  • thenastyranch
  • kavyap
  • osvaldo12
  • provamag3
  • ethstaker
  • GTA5RPClips
  • khanakhh
  • Durango
  • tester
  • normalnudes
  • cisconetworking
  • modclub
  • cubers
  • anitta
  • Leos
  • megavids
  • JUstTest
  • All magazines
    •