Had a serious database failure earlier due to cascading bad decisions. I started deleting a bunch of old data, rather than using #tootctl. This led to a bunch of errors in the #pgsql#docker container.
I moved the database into bare metal using a #nuc with 32GB of RAM and 4 cores. I'm going to use this for other services too, and it was on my roadmap of transition. #nextcloud will be moving to that as well.
I absolutely love being a sysadmin to this server and many others, even if it means during my holiday I need to be the maintenance guy. During my Christmas holidays so far I've
Improved one clean up script that had an error which temporarily made one of the smaller disks fill up too soon
Fixed a nginx block for a domain that didn't allow renewing a Let's Encrypt HTTPS certificate for a certain domain after database migration and re-release of a web service
Repaired an old MySQL database manually, because of a very old MyISAM corruption bug
Some people find this stressful, but for me it's like taking the trash out, these are chores I'm very accustomed to. Most of the weeks they require nothing, but for some reason it's the holidays when some things just happen. That's life.
I'm very glad Mastodon stack is so sophisticated compared to some of the older stacks I'm running also I thank myself and my experience from last year that I really built it almost fool-proof and big-enterprise style. This is why no big hiccups present here, yet. 24/7 backup system is in place, too.
:skull360: Heads up! I need to rescale the database drive soon. It’s growing and I want the disk to be able to hold more data. The current max size is about 40 GB and I will grow it to 500 GB to make it more future proof.
This means I have to shut down my instance for this operation to make sure there will be no data corruption. The downtime will be probably only mere minutes, but there will be visible downtime. I’ll let you know the exact time for this to take place.
Database operations always make me kinda anxious for some reason despite the fact there has not been a single issue with them for me in the past.
Just set up a log watcher via Datadog for my Mastodon server cluster, because it's the most pretty and easiest for me in this situation. Works amazing and now I will receive a notification from even a tiniest anomaly!
¿Es muy raro que me suban las endorfinas y casi me alegre cuando mi madre me dice que no funciona la web de Nextcloud donde compartimos fotos de mis hijas y así tengo algo más que arreglar? #Nextcloud#SysOp#DNS
I here by declare I have opened my server to the public. I'm going to keep it rather small. If you like an account you'll have to accept things are prone to change as this server is experimental and hosted by one person.
I will take no pressure from this nor blame for anything. I will take it chill. I will cherish you and appreciate you, but won't necessarily wish for any influx. Let's see how it goes. :bunhdheart:
@mlaursen There is a little bit of a history behind this Mastodon Instance. SMUG stands for "Sequoia Macintosh Users Group" It was a BBS operated using #FirstClass server in #HumboldtCounty, #California. It went offline a very long time ago. I found the original domain (htan.org) bought it and now we are here. ❤️ 😎 Attached are some images from the original #sysop SMUGgler's was the first online experience I had at home as a young little nerd. 🤓
Hi! After studying chemistry, I’ve spent the last twenty five years in IT, first as a #Web1.0 programmer and #sysop during the 90’s, then in #itsupport and #dataanalyst roles this century.
I read “My Side of the Mountain” and “The Mysterious Island” as a kid and have always been fascinated with a diy approach to civilization: #camping, #gardening, #hamradio, and #brewing.