645 days after setting up my own Mastodon instance, I finally tackled tuning my Postgres database config last wk & today setup multiple Sidekiq Processes after my queue got to 10K for the second time this wk
I read the guides multiple times over those 645 days but it never clicked in my head. I was afraid to mess with Sidekiq because I didn't want to brick my instance.
Then today, during the thunderstorm, the old grey cells clicked in and I understood.
2/ I'm sure others can add or even help me with anything I got wrong, or how I can improve this, but here is what I did today out of urgency to scale my Mastodon Sidekiq processes and jobs after getting backlogged twice this week.
@paul@trendless@shanie Don't know if you already know, but when you specify '-q push -q pull' it will do push first, and if there's no more push to process, start with pull.
Also from your notes it seems you now only have default, push and pull queues. If that's the case, you're missing the scheduler, mailers and ingress.. or I misread your note.
@ruud@trendless@shanie
I did some tweaking since I wrote that, but I left the original mastodon-sidekiq.service enabled, which I copied as my template, so it still does the scheduler, mailers and ingress, plus I have an ingress with 25 threads.
I am basically single user instance, so, I hope I am pushing my server out before I pull (mostly federated) stuff in.😉 Can you confirm that is how it is?
@paul@trendless@shanie Yes it looks OK. You could give a second queue to the ‘default’ and ‘ingress’ processes so they do something when the default and ingress queues are emtpy.
I have way too many configured now, but who knows maybe one day there might be a new user influx and I’ll be ready :-)
I'll be honest. I am using my ElasticSearch powered Mastodon search more than Google, et al, now.
I followed #Flipboard's relay, which adds a lot of content from their magazines that link to sites as well as some other relays. I think they are the culprit for my now fixed long pull queues. Today there was no backlog. First time since I followed their relay.
Soon I'll need to work on database optimization, which will probably move it from this server.
It finally motivated me to tackle a server 'honey-do' project that I had been putting off for 645 days.😂
I'm just thankful I am finding neat and relevant content from #Flipboard's relay when I search, and I do several times a day, even if I don't see the Flipboard post when it comes through the firehose.✋ 👊
Add comment