Been setting up a Yunohost sever and really been enjoying the process/experience . The one snag I’ve had is setting up a personal Lemmy sever and the Yunohost fourms seem to be dated. Just wondering if someone might know the best place to look for solutions. Thank you, and have a good day.
What I’m using is Text Generation WebUI with an 11B GGUF model from Huggingface. I offloaded all layers to the GPU, which uses about 9GB of VRAM. With GGUF models, you can choose how many layers to offload to the GPU, so it uses less VRAM. Layers that aren’t offloaded use system RAM and the CPU, which will be slower.
Didn’t know that. I’m actually the sole moderator of a community I made on lemmy.ml, so that’s good to know. I do still have my old account on lemmy.ml as a moderator too tho.
Yunohost Lemmy qeustions:
Been setting up a Yunohost sever and really been enjoying the process/experience . The one snag I’ve had is setting up a personal Lemmy sever and the Yunohost fourms seem to be dated. Just wondering if someone might know the best place to look for solutions. Thank you, and have a good day.
AI is the future (lemmy.world)
Self hoating an LLM for research
I am a teacher and I have a LOT of different literature material that I wish to study, and play around with....
[Meta] Looking for Moderators
/c/opensource is currently unmoderated because all the existing mod accounts are inactive....