In #homeassistant, using #nodered to make an API call to a #llamacpp server running #mistral 7B model. I create a prompt that asks it to summarize all the data in my house from the sensors. The results are pretty impressive for such a little model. Now I get a customized rundown, Jarvis style.
Useful? Probably not. But cool as hell. :cool_skelly:
I know that LLM and AI are kind of dirty words in the app dev community and I get the reasons why. Even if you just keep living your life, folks like Dropbox (https://boingboing.net/2023/12/14/dropbox-is-sharing-users-files-with-openai-heres-how-to-opt-out.html) and GitHub are using your data to train their LLMs. I think that there is an opportunity in ethically sourced AI, though. Even if you disagree with that statement, there is a "know your enemy" argument.
That's the reason why I have been trying to learn this shit, lately. ...
... I can get llama.cpp to answer simple questions. When I ask it for the meaning of life, it says "The meaning of life is a philosophical question that has been debated throughout history. It varies from person to person based on their beliefs, values, experiences, and worldview. As an AI, I don't have personal beliefs or emotions; therefore, I cannot provide you with a definitive answer."
I still need to figure out how to finetune it and how to connect it to a web interface.