Come funzionano gli #LLM, spiegato senza matematica
Da dove proviene l’apparente intelligenza di questi modelli. In questo articolo, cercherò di spiegare in termini semplici e senza utilizzare la matematica avanzata come funzionano i modelli di testo generativi, per aiutarti a pensarli come algoritmi informatici e non come magia.
@luis_in_brief Also, this is one of two FirefoxOS devices I owned. It took extra digging in the garage to find the orange one, which was obviously critical for the bit.
I created a multi-needle in a haystack test where a randomly selected secret sentence was split into pieces and scattered throughout the document with 7.5k tokens in random places. The task was to find these pieces and reconstruct the complete sentence with exact words, punctuation, capitalization, and sequence. After running 100 tests, llama3:8b-instruct-q8 achieved a 44% success rate, while llama3:70b-instruct-q8 achieved 100%! #LLM#AI#MLhttps://github.com/chigkim/haystack-test
For inference, the best option right now is llama.cpp with quantized LLM in GGUF format. There are several high-lever wrappers around llama.cpp that makes it easy to use: ollama, vllama...
For inference with very big LLM and very small RAM, the only option is airLLM: it's slow, but you can run llama3-70b
For finetuning quantized LLM with LoRA, the only option afaik is also llama.cpp (look for "finetune"). It's a work in progress but usable and promising!
Beyond the brain: Our intelligence leverages the power of culture and language. Channeling Ted Underwood and Francios Chollet, I argue that language models, despite their biases and lack of understanding –– are important tools for thinking. 🗣️🌍💡 cc @TedUnderwood https://leviathan.substack.com/p/beyond-the-brain
Ganz schön arrogant, wie einige #KI bzw. #LLM verteufeln.
Nicht jedem fallen die Worte aus den Fingern wie frühreife Früchte, für viele ist das Schreiben ein zähes, träges Mäandern.
Und wenn KI da bei der Textarbeit hilft und diesen Menschen mehr Partizipation und Produktivität und einfach ein besseres Gefühl ermöglicht, ist das toll!
Anthropic released an iOS app for their Claude 3 LLM.
I’m past the stage that dismisses LLMs. Some variant will be a useful tool for me. For various tasks. Some I haven’t thought of yet. I’m currently using them as research assistants on topics I’m writing about. To see if detailed prompts (several hundred words with topic headings etc) get responses that include things I’d overlooked. I don’t use any generated text directly.
I might use Claude as a tutor for some studying I plan. #LLM
@knitter Yikes. Have we stumbled on a Brexit benefit! It’d be a first! Use the web page directly via VPN if you need to. The iOS app experience is the same as the web app. https://claude.ai/
An #LLM#AI trained on playlists and music reviews who talks with a sneer like a local record store clerk who just doesn't have time for your pedestrian tastes. #ideas But if it must, it will make recommendations.
The amazing thing about LLMs is how much knowledge they posess in their small size. The llama3-8b model, for instance, weighs only 4.7GB yet can still answer your questions about everything (despite some hallucinations). #llm#ai#ollama#llama3
I am trying to build a RAG with LLAMA 3 and... getting really crazy with the strange formats I get in the response....
Not only the response, but additional text, XML tags... #Llama3#LLM#RAG