[RESOLVED] lemmy.ca sends every activity out 3 times?

Hi. Just looking at what lemmy.ca sends my server:

From nginx log:


<span style="color:#323232;">your.ip.address - - [20/Feb/2024:03:48:47 +0000] "POST /inbox HTTP/1.1" 200 0 "-" "Lemmy/0.19.3; +https://lemmy.ca"               
</span><span style="color:#323232;">your.ip.address - - [20/Feb/2024:03:48:49 +0000] "POST /inbox HTTP/1.1" 400 0 "-" "Lemmy/0.19.3; +https://lemmy.ca"         
</span><span style="color:#323232;">your.ip.address - - [20/Feb/2024:03:48:57 +0000] "POST /inbox HTTP/1.1" 400 0 "-" "Lemmy/0.19.3; +https://lemmy.ca"              
</span>

Dumping the JSONs reveals 3 duplicate files, all containing:


<span style="color:#323232;">"id": "https://lemmy.ca/activities/create/0968bf93-ac05-4178-bf3f-28cdc65fc338"
</span><span style="color:#323232;">"actor": "https://lemmy.ca/u/HairyOldCoot"
</span><span style="color:#323232;">"object": {
</span><span style="color:#323232;">  "id": "https://lemmy.ca/comment/7454502"            
</span><span style="color:#323232;">  "content": "That was an episode of Sliders."
</span><span style="color:#323232;">  "published": "2024-02-20T03:48:45.810068Z"
</span>

These are 3 Creates, not 1 Create and 2 Updates or anything. I’ve chosen a comment 'cos that’s public info, but it’s the same for all other activity too (Votes, Deletes, etc).

lemmy.ca isn’t alone in this. endlesstalk.org sends everything twice (info posted here), as does lemmy.sdf.org and mander.xyz, but lemmy.ca is the only one I’ve seen send stuff 3 times. Other instances (e.g. aussie.zone, feddit.nl, lemm.ee, lemmings.world, lemmy.world, programming.dev, reddthat.com, sh.itjust.works, slrpnk.net) don’t, but there’s nothing obvious like lemmy version numbers to indicate the difference.

Whilst I’d prefer you didn’t send me stuff 3 times, I’m mostly mentioning it because I doubt it’s good for your own resources to send every activity by every lemmy.ca user out multiple times. Also, the OPs of my community are from Mastodon, so I imagine you’re sending them the same message 3 times too …

Sal,
@Sal@mander.xyz avatar

Thank you for the heads up… In my case, I am running a single lemmy docker container for mander.xyz, so the reason for the duplicates is not yet clear to me. I will investigate.

nutomic,
@nutomic@lemmy.ml avatar

Did you find out anything unusual that might be causing the duplicate activities? We have an issue for this now so we can hopefully resolve it.

mp3, (edited )
@mp3@lemmy.ca avatar

Hi, /u/Shadow recently modified our configuration by adding additional containers (3 lemmy, 3 lemmy-fed, 3 lemmy-ui) so that we can restart each of them without causing an outage.

lemmyverse.link/lemmy.ca/post/15548660

An issue was also discovered yesterday (9 hours ago approx) where all three federation instances were using index=1 in the docker-compose. Maybe the issue is already fixed, according to the timestamps you have vs when we applied some changes?

freamon,

Thanks. This seems a likely answer - running 1,2 or 3 containers would explain the differences across instances. I hope it can be resolved though.

Shadow, (edited )
@Shadow@lemmy.ca avatar

This should have been fixed yesterday before your post, although it was taking some time to catch up. Are you still seeing this behavior from us?

Thanks for posting this!

freamon,

I’ve turned logging back on, so I’ll let you know once someone from lemmy.ca interacts with !tails

Shadow, (edited )
@Shadow@lemmy.ca avatar

Thanks, I just sent a subscribe

I need to dig into the lemmy code, but I think the federation index / count system may be broken.

freamon,

Oh, well I only got one copy each of your Subscribe / Unsubscribe / Subscribe actions, so that’s good! Many thanks.

Shadow,
@Shadow@lemmy.ca avatar

Awesome, glad to hear it. Thanks again for letting us know about the problem.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • lemmy_ca_support@lemmy.ca
  • DreamBathrooms
  • magazineikmin
  • vwfavf
  • thenastyranch
  • Youngstown
  • slotface
  • Durango
  • ngwrru68w68
  • rosin
  • kavyap
  • tsrsr
  • PowerRangers
  • osvaldo12
  • khanakhh
  • normalnudes
  • hgfsjryuu7
  • cubers
  • everett
  • cisconetworking
  • InstantRegret
  • mdbf
  • tacticalgear
  • ethstaker
  • tester
  • modclub
  • GTA5RPClips
  • Leos
  • anitta
  • All magazines