befreax, 2 months ago

This has been fun to learn about #LLMs, #RAG and their behavior on modern #infrastructure; I just push my simple #rust based #service that uses Mistral 7B for inference that is (hopefully) easy to instrument: https://github.com/tmetsch/rusty_llm

An here is the matching image generated by #Dall-E a rusting llama being inspected while being in mistral winds.