@lodion@aussie.zone avatar

lodion

@lodion@aussie.zone

This profile is from a federated server and may be incomplete. Browse more on the original instance.

lodion,
@lodion@aussie.zone avatar

As Nath posted, the domain was registered June 8th 2023. Lemmy was set up on the domain that same day.

That’s a year that has flown by…

lodion,
@lodion@aussie.zone avatar

I disagree. Its deliberately inflammatory and offensive. I only leave the post here as responses such as this one have value.

lodion,
@lodion@aussie.zone avatar

I’m locking this post. There has been some good discussion within the comments, but I don’t believe the linked article is in good faith, and the discussion here is heading off the rails.

lodion,
@lodion@aussie.zone avatar

Yep, thats what it means. What is the swap monster? Seems to be a mix of the lemmy backend and the associated Postgres DB. Its likely the Postgres config is sub optimal and my fault, but honestly given the other issues the lemmy code has had in the past I’m happy to leave the daily restart in place.

lodion,
@lodion@aussie.zone avatar

Current kitty is enough for the next ~7 months, thanks to some generous larger donations in the last few days. Ongoing monthly contributions currently cover about 50% of the monthly operating costs. Early on monthly donations exceeded monthly costs… leading to the current surplus. Between the surplus, and the existing monthly donations, we won’t hit the red for… 12 months?

Total costs are ~$85AUD per month for the server and object storage. I can cover that myself if needed.

tldr; finances are fine 🙂

lodion,
@lodion@aussie.zone avatar

TPAs?
CF is best for static content such as images, by default that’s almost all it caches. We’ve had some popular images cached and save us terabytes of egress traffic.

lodion,
@lodion@aussie.zone avatar

All good, I’ve been very busy recently… New job, travel for work, family etc. Will post an update soon™.

lodion,
@lodion@aussie.zone avatar

Nerdy stuff… aussie.zone/post/9041040

lodion,
@lodion@aussie.zone avatar

Cheers, Phil. Appreciate it.
Seems to be the lemmy backend and its associated Postgres DB.

lodion,
@lodion@aussie.zone avatar

Appears legit now. Did you grab a screenshot?

lodion,
@lodion@aussie.zone avatar

☹️

I’ll see what I can see…

lodion,
@lodion@aussie.zone avatar

Ok, for some reason practically all instances were flagged as “dead” in the database. I’ve manually set them all to be requeued… server is now smashed as it attempts to updated the ~4000 instances I’ve told it are no longer dead. See how this goes…

lodion,
@lodion@aussie.zone avatar

Seems to have sorted it for the most part… not sure what caused it, will do some more digging.

lodion,
@lodion@aussie.zone avatar

reply

lodion,
@lodion@aussie.zone avatar

Ok, something is busted with the lemmy API endpoint that shows current federation state. It is currently showing nearly all remote instances as dead:
https://aussie.zone/pictrs/image/2b58f6a0-f9a8-4531-ba65-895697f85f5c.png

But “dead” instances are still successfully receiving content from AZ, and sending back to us.

lodion,
@lodion@aussie.zone avatar

Ok, there’s more to this than I first thought. It seems there is a back end task set to run at a set time every day, if the instance is restarting at that time the task doesn’t run… this task updates the instances table to show remote instances as “seen” by AZ. With the memory leaks in 0.19.1, the instance has been restarting when this task is running… leading to this situation.

I’ve updated the server restart cronjob to not run around the time this task runs… and I’ve again manually updated the DB to flag all known instances as alive rather than dead.

Will keep an eye on it some more…

For anyone curious, two of the bugs that are related to this:
github.com/LemmyNet/lemmy/issues/4288
github.com/LemmyNet/lemmy/issues/4039

lodion,
@lodion@aussie.zone avatar

Excellent, hopefully that is included in 0.19.3 that came out overnight… will upgrade shortly.

lodion,
@lodion@aussie.zone avatar

Cool, we’re on 0.19.3 now.

lodion,
@lodion@aussie.zone avatar

0.19.3 still has the memory leak, I’ve left Lemmy to restart once a day to mitigate this. Federation seems to be fine for the last week after some minor changes and the upgrade.

lodion,
@lodion@aussie.zone avatar

You’ve got a way with words… are you a writer? :)

lodion,
@lodion@aussie.zone avatar

AEST? That would be the daily server snapshot. I’ll see if I can bring it forward, trouble is there are daily Lemmy DB tasks etc running earlier that I don’t want to disrupt .

I’ll see what can be done …

  • All
  • Subscribed
  • Moderated
  • Favorites
  • megavids
  • thenastyranch
  • magazineikmin
  • InstantRegret
  • tacticalgear
  • osvaldo12
  • Youngstown
  • everett
  • slotface
  • vwfavf
  • rosin
  • mdbf
  • kavyap
  • DreamBathrooms
  • provamag3
  • ngwrru68w68
  • Durango
  • cubers
  • ethstaker
  • khanakhh
  • tester
  • modclub
  • GTA5RPClips
  • anitta
  • cisconetworking
  • normalnudes
  • Leos
  • JUstTest
  • All magazines