Behind search for masto is a resource-hungry #java application called #elasticSearch. Sure you might be able to get Masto to run on a #raspberrypi4, but you won't be running elastic search with it.
Having recently moved my instance to a much more powerful system, I now run ES, and WOW what a difference being able to search post makes! Soooo many times I wanted to reference a toot that had scrolled by but had no way of finding it. Now I do!
Even though I've put some serious resource limits on ES, it's bar none the most resource intensive service running (out of around 25).
Please consider sending a few bucks to the #MastodonAdmin of your instance. Better yet, if you can afford it, sign up for a monthly donation. They need it.
ES|QL is a great new query language for #Elasticsearch, and my team is building integrations into the client libraries. Here's an overview of our #PHP integration that lets you turn your results into PHP objects.
After reading a massive tome about #ElasticSearch earlier this week I realised it was complete overkill and just used the full-text capabilities of #PostgreSQL instead.
Currently PieFed has 46,000 posts and results are fast. It'll be interesting to see how well it copes when there are more posts. Anyone want to make a guess when it'll bog down?
@jillesvangurp Exactly, a server is there to serve stuff. If one process eats all the memory and locks up the entire system, that's an even bigger problem. I rather have a single service killed than having ALL services fail.
Just to make that clear, there wasn't plenty of memory. The machine has 24 GB and ElasticSearch wanted all of it.
Great to see more people catching on to ClickHouseDB. We’re using ClickHouse at @honeybadger to power our upcoming logging/observability tool (Honeybadger Insights).
We’re also benchmarking a replacement backend for #Elasticsearch. Looks like quite a performance gain so far!
Will hopefully have more to share soon, but in the meantime we discussed this on the latest episode of @FounderQuest. Give it a listen:
I'm planning to write an updated homelab guide on my blog this year but I think I'm about to rebuild some parts for a new purpose 😅
It might be time to try out OpenCTI given what I do in my lab should be representative of what I do during < dayjob >. That also means I need to tear down Wazuh and configure an ELK stack instead (resource constraint). #Homelab#ELK#CTI#ThreatIntel#elasticsearch
@ironicbadger That is true. I could at least focus on the infrastructure changes switching from VirtualBox on a shared desktop to a standalone Intel NUC running Proxmox.
Q for people who have used Elastic Search, esp. for a Mastodon instance: how should I configure it to use less memory (while still having enough)? Right now it seems to eat as much as it wants (~4 GB)...
It's a single-user instance, so the total data size it reports is 40 MB now.
Delighted to let everyone know that https://fediverse.au has been successfully upgraded to 4.2.5 of #Mastodon software (thank you, Claire!).
Additionally, #ElasticSearch has been installed for a better user experience.
Fediverse AU is exclusively for #research organisations and #university schools, centres and #faculties in the Australasian region - as most Mastodon instances don't allow these sorts of corporate accounts.
Please let me know if you experience any issues or glitches.
(you will probably need to add -u user:password to curl)
also if you want to start from scratch, check if you deleted the data directory which is specified in /etc/elasticsearch/elasticsearch.yml and restart elastic afterwards
@fell@moira you can build it yourself. I doubt that will solve your issue though
how do you install it (what's the binary) and the directory layout there? if you keep having leftovers from previous installations, we need to remove them (explicitly the data directory) to start a fresh bootstrapping