notfromhere,

I get what the author is talking about with losing trust in your services when they are unavailable. I have found that a Raspberry Pi 3/4 cluster running k3s utilizing NAS storage has very good uptime and honestly I think my internet service goes down more than my homelab services.

jax,
@jax@lemmy.cloudhub.social avatar

Thanks! I find most of the issues occur during upgrades to services, but that is to be expected.

My internet service is usually more of an issue than most services I run. Though some things take longer to get tweaked and running well and that can cause issues.

notfromhere,

Upgrades to services - that’s why I run dev namespaces and copy over my production shares to dev and utilize zfs snapshotting.

I haven’t set up testing yet and only just started with prometheus monitoring but so far things run pretty well.

jax,
@jax@lemmy.cloudhub.social avatar

Oh, dev namespaces are a good idea. Do you have a dev domain then too?

notfromhere, (edited )

I toyed with dev domain but ended up using namespace.tld and postfixing -dev to my namespace so it works out to service.tld and service-dev.tld.

jax,
@jax@lemmy.cloudhub.social avatar

Ah okay that makes sense, you’re using the internal cluster domain to route to services

notfromhere,

I have automated traefik to route the traffic, it sets the dns and ingress route. I’m also doing as you suggested for service to service connections.

jax,
@jax@lemmy.cloudhub.social avatar

That makes sense!

Have you played with anything like Istio to secure in-cluster communications? I think Hashicorp Consul can do something similar to encrypt service to service communications.

notfromhere,

I looked into it but I felt at the time it was too complex, maybe I’ll look at it again. Currently I am using wireguard for all cluster node-to-node traffic. It seemed like a reasonable tradeoff at the time, but it is at the network layer instead of application, so I really should revisit that at some point.

jax,
@jax@lemmy.cloudhub.social avatar

Yeah it very adds some extra complexity and it’s more important for if you are hosting in public clouds anyways IMO.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • homelab@lemmy.cloudhub.social
  • mdbf
  • DreamBathrooms
  • thenastyranch
  • ngwrru68w68
  • tacticalgear
  • magazineikmin
  • Youngstown
  • khanakhh
  • rosin
  • slotface
  • InstantRegret
  • everett
  • kavyap
  • cisconetworking
  • JUstTest
  • cubers
  • modclub
  • GTA5RPClips
  • ethstaker
  • osvaldo12
  • Durango
  • provamag3
  • normalnudes
  • tester
  • anitta
  • megavids
  • Leos
  • lostlight
  • All magazines