It should be possible using script to execute on alarm = /your/custom/remediation-script…netdata.cloud/…/agent-notifications-reference. I have not experimented with this yet, but soon will (implementing a custom notification channel for specific alarms)
restarting a service if it isn’t answering requests
I’d rather find the root cause of the downtime/malfunction instead of blindly restarting the service, just my 2 cents.
I use netdata (the FOSS agent only, not the cloud offering) on all my servers (physical, VMs…) and stream all metrics to a parent netdata instance. It works extremely well for me.
Other solutions are too cumbersome and heavy on maintenance for me. You can query netdata from prometheus/grafana [1] if you really need custom dashboards.
I guess you wouldn’t be able to install it on the router/switch but there is a SNMP collector which should be able to query bandwidth info from the network appliances.
I agree that desktop/ATX tower PCs are the most useful form factor, you can stuff all your old junk hardware in there and offer it a second life without much investment.
However with current electricity prices buying more power efficient hardware can be a better medium-term investment. 1kWh bills at 0.2516€ currently where I’m at (~EU average price), assuming an average power consumption of 50W this gives you (50×24×365)/1000×0.2516=110€/year. At this rate a 200€ investment in hardware would pay for itself in 2-3 years.
Buying a <100€ setup is not worth it for general purpose servers in my opinion, it will either be underpowered or power hungry.
My current solution is to to run all my services in KVM (libvirt) VMs on my beefy desktop computer which is already on most of the time anyway. Best of both worlds.
If I had to redo everything I would probably buy a NUC/mini-PC with a good CPU, 64GB RAM and low power consumption, stash a single huge SSD in there, migrate my VMs there and call it a day. But this is not a cheap setup.
Netdata can also expose metrics to prometheus which you can then use in Grafana for more advanced/customizable dashboards learn.netdata.cloud/docs/…/prometheus
github.com/chriswayg/ansible-msmtp-mailer/…/14While msmtp has features to alter the envelope sender and recipient, it doesn’t alter the “To:” or “From:” message itself. When the Envelope doesn’t match these details, it can be considered spam
Oh I didn’t know that, good to know!
The proposed one-line wrapper looks like a nice solution
(the only thing I changed from the defaults in the aliases file is adding the last line)
This makes it so all/most system accounts susceptible to send mail are aliased to root, and root in turn is aliased to my email address (which is the one configured in host/user/password in msmtprc)
Edit: I think it’s actually the auto_from option which interests you. Check the msmtp manpage
Don’t mind him. He’s always there ranting about who knows what whenever software he dislikes is mentioned. Lookup his comment history for more of the same.
Easiest method to summon him is to mention Nextcloud and Proxmox in the same sentence.
Usually you would have a second DNS resolver configured in /etc/resolv.conf (or whatever name resolution config system you are using, resolvconf, systemd-networkd, etc). The system will fall back to this resolver if the first resolver fails to respond (and/or replies NXDOMAIN, I’m not sure. The exact order and fallback conditions may vary depending on which system you use). This can be another dnsmasq instance, a public DNS resolver, your ISP’s resolver, etc. This allows at least basic DNS resolution to work before your dnsmasq instance comes back up.
I would also add automatic monitoring for dnsmasq (either check that the service/container is running, or check the TCP connection to port 53, or check that DNS resolution is working for a known domain, etc)
Not an answer but still relevant: I actively avoid enabling unattended-upgrades for third-party repositories like Docker (or anything that is not an official Debian repository) because they don’t have the same stability guarantees, and rely on other upgrade notification methods instead.
how bad of an idea is this to run a DNS in docker and use it for the host and other containers?
Personally I would simply install dnsmasq directly on the host because it is one apt install and a configuration file away. Keep it simple.