TheDude

@TheDude@sh.itjust.works

Entrepreneur, developer, seasoned system and network engineer, aspiring hobbyist pilot, husband and father.

This profile is from a federated server and may be incomplete. Browse more on the original instance.

TheDude,

There was a little hiccup this morning unrelated to the migration tonight. The Lemmy services like to be restarted every once in a while. Once we move to the new hardware I’ll be able to look into implementing a better logging system and hopefully be more proactive when situations like these happen.

TheDude,

The biggest consumer of storage on this instance is related to the image hosting which we use an external object storage provider for. The second is the database which is no were near the 2TB capacity. 1TB SSDs are cheaper than 2TB SSDs and I also didn’t want to spend more than I needed. As other mentioned if we need more space or IOPs in the future, I could accomplish this by adding more drives as a quick fix. This server does not support NVME unless I leverage its PCIe ports but I don’t plan on doing that. By the time this instance gets to the point where 10 SSD drives just isn’t cutting it anymore I’ll probably have come across another opportunities on getting a new server with better NVME support.

Trying to sign up on sh.itjust.works l, no verification email sent

Hello, I am trying to sign up on sh.itjust.works, and I filled out the sign up form, but when it told me it would send a verification email, it never actually sent it. I tried signing up again and am told the email already exists. I try to sign in and nothing happens, I dont get an error it just doesnt do anything. I have...

TheDude,

The big email email providers are not liking the amount of email coming from the server ip. The permanent fix is to use a mail sending service. I do plan on making that adjustment soon.

In the meanwhile send me a dm with your username and email address used and I’ll verify the account for you.

Thanks

TheDude,

Hey all, I’m always excited to upgrade to the latest versions that become available. Unless the upgrade is related to security I will tend to wait a few days to give other instances a chance to share their experiences. Before upgrading I usually go over the issues that have been submitted to help understand the risks and solutions to mitigate them.

That being said the new version does bring in a whole bunch of performance gains (mostly related to the database) that i’m looking forward to.

Lemmy.world is due to upgrade later today and I plan on upgrading shortly after them (either Monday or Tuesday).

I will put up a notification up prior to the upgrade.

Thanks

TheDude,

Hi all,

I somewhat explain some of the recent downtime here sh.itjust.works/comment/1341394

Postmortems are really great when they are done well and you have a team of sysadmins to collaborate with. As you probably know most instance operators and moderators are volunteering their time (and sometimes money) to keep these instances online. As the lemmyverse matures in age and development things will get better. In the meanwhile there will be some hiccups along the way.

When I have some extra time I plan on sharing more details about all of this. In the meanwhile check in on our chat. I’m pretty active there and can always provide additional insights on whats going on during an outage.

TheDude,

Hey all,

There has been a few issues over the last couple of weeks. Here’s a rundown of some of the issues we’ve had.

  • Broken Images: A few weeks ago we had issues with broken images. This was due to me migrating our local image storage to object storage (like AWS S3). When I did the switch it broke all old images as they needed to be migrated. Pictrs at that time did not support concurrent uploads which means the migration would have taken days or weeks to complete during which time the image service would be offline. Instead I waiting for a newer version to be released that supported concurrent connections and did the migration in about 30-40 minutes one evening.
  • 5xx server errors: Some of you may have experience a lemmy page with an error code on it. This was due to me trying to implement an additional proxy to shield and mitigate future risks. While rolling this out I hit a few blockers that caused downtime as we worked to rectify it. I’m glad to say that as of Today this has since been implemented.

In the addition to the above, the lemmyverse and especially this instance has been under bot attack almost daily. These bot attacks are eating resources and causing a query floods.

Lemmy is still very young and in its early phases. In time these issues will slowly go away.

P.S you don’t have to worry about me leaving you guys hanging.

TheDude,

Honestly these type of scat spamming has been really discouraging on me. I spend time and money so that the community can have a reliable and nice instance to call home, only to have a few (not sure what to call them) people try to ruin it for everyone. If these people are reading this comment, just know that every time you make these kinds of unwanted posts you are taking my time away from my family and young kids to address the reports.

My nights lately have been mostly used for coding some extra moderation tools that will moderate these in the future.

I sincerely apologize for all of you who have been exposed to literal shit images over the last week. I’m working as fast as I can and will implement something soon.

In the meanwhile i’ve added a few new admins to help with this moderation.

TheDude,

I’m actually quite amazed by how little would be needed to keep this instance fully funded and even provide other services at the same time. If just 1/4 of the users gave 25cents per month we would have enough money to cover the hosting, new hardware and some paid development/administrative work! The problem however still resides with not having a fiscal host to be the entity needed to accept donations.

Working on some alternatives at the moment.

TheDude,

Offloaded the images to an S3 compatible host. Will take a bit but I will be migrating the older images shortly

Is s.itjust.works affected by the XSS Vulnerability?

There’s plenty of posts on the topic about Lemmy.world being compromised, followed by the exploit being tracked back to an XSS exploit that I believe works on instances with custom emojis enabled. Many instances have been quick to jump on this such as feddit.uk and Behaw which took itself down temporarily....

TheDude,

Hey all,

As others mentioned we did not have custom emojis so we were not affected by this particular attack. I have since upgraded our UI to 0.18.2-rc.1 which mitigates this XSS vulnerability.

TheDude,

The vulnerability appeared to be from a custom emoji that they were running. SJW does not use any custom emoji so we should not be affected. In either case lemmy.world has now been restored and is back online. I’ll keep an extra eye on this instance until the patch gets released shortly.

TheDude,

Hey all, I’ve reopened the ability to create new communities. You should now be able to create new communities.

[Help wanted] - Seeking mods for The Agora

Since making the announcement last night to become community driven, The Agora has quickly become an active place full of discussions, proposals and voting. In order to keep things going I'm looking for volunteers to help facilitate the conversations and apply the rules that have been determine by this community....

TheDude,

Looks like this decision is decided already but I figured I'd put in my input. Given this instance has an open registration policy, nothing is stopping someone from another instance to create an alternative just for voting on this instance. This individual might align and contribute positively to the fediverse and have really great ideas and contributions to discussions here. However because they use their alt account here on sh.itjust.works only for voting, their vote might get dismissed due to poor account reputation (another issue that I believe was already brought up in another post). The fediverse is meant to be a decentralized community and by forcing people to need to join this community to vote promotes centralization which I believe is the opposite of what the fediverse is trying to accomplish. I guess for now I'll hold off on casting my vote until the community determines what criteria should be considered when counting a vote.

TheDude, (edited )

Captchas have been enabled on this instance.

TheDude,

This is the way.

Btw thanks guys

Are artists and creators not welcome in this community? (sh.itjust.works)

I decided to do a bit of a dive into this platform, and while I think it is an interesting idea, I am yet to find a space on Lemmy where I am welcome. I am an artist. I have a Patreon that pays my bills and puts food on the table. Generally, I give the majority of my stuff away for free to the community, but that usually comes...

TheDude, (edited )

I agree with all of you and think artists and others alike should be able to share their work and allows other to contribute to their passion. Should we just remove the No Ads from the rules? Maybe the answer is to modify the rule. I just to make it clear that I don't want users aimlessly ad spamming the communities.

Beehaw defederating update - sh.itjust.works

Like many of you, I woke up this morning to discover that our instance, along with lemmy.world, had been unexpectedly added to the beehaw block list. Although this development initially caught me off guard, the administrators at beehaw made an announcement shedding light on their decision....

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • JUstTest
  • mdbf
  • InstantRegret
  • ethstaker
  • magazineikmin
  • GTA5RPClips
  • rosin
  • modclub
  • Youngstown
  • ngwrru68w68
  • slotface
  • osvaldo12
  • kavyap
  • DreamBathrooms
  • Leos
  • thenastyranch
  • everett
  • cubers
  • cisconetworking
  • normalnudes
  • Durango
  • anitta
  • khanakhh
  • tacticalgear
  • tester
  • provamag3
  • megavids
  • lostlight
  • All magazines
    •