I know that LLM and AI are kind of dirty words in the app dev community and I get the reasons why. Even if you just keep living your life, folks like Dropbox (https://boingboing.net/2023/12/14/dropbox-is-sharing-users-files-with-openai-heres-how-to-opt-out.html) and GitHub are using your data to train their LLMs. I think that there is an opportunity in ethically sourced AI, though. Even if you disagree with that statement, there is a "know your enemy" argument.
That's the reason why I have been trying to learn this shit, lately. ...
... I have tried running #Llama 2 (https://ai.meta.com/llama/) locally and I haven't been able to figure it out yet but it is the reason why I decided to replace my 2017 MBP with a 2021 MBP (with double the RAM and an M1 Pro instead of an i7).
I did have some success with #llamafile (https://github.com/Mozilla-Ocho/llamafile), though. Mozilla released something that lets you package a set of weights as a single multi-gigabyte executable that can run on macOS, Windows, Linux, FreeBSD, OpenBSD, or NetBSD. ...
... I can get llama.cpp to answer simple questions. When I ask it for the meaning of life, it says "The meaning of life is a philosophical question that has been debated throughout history. It varies from person to person based on their beliefs, values, experiences, and worldview. As an AI, I don't have personal beliefs or emotions; therefore, I cannot provide you with a definitive answer."
I still need to figure out how to finetune it and how to connect it to a web interface.
@joe You're running the web interface in these screenshots! As for the models, if you have the ability to run llama.cpp directly (as opposed to running it through llamafile) you can find models in the appropriate format on huggingface -- https://huggingface.co/models?search=gguf . If your ram is beefy enough, I would absolutely recommend trying out the Mixtral 8x7b's, like https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF .
Add comment