Does anyone have experience + feedback with using LLMs on your technical docs as a customer-facing help chatbot? How effective is it? What are the caveats? #ai#llm#llms
I downloaded and tried Dolphin 2.7 Mixtral 8x7b today. So far, it's performing well. It's not too slow, averaging around 7.5 tokens per second. Tomorrow, I plan to test its performance in different topics. Perhaps I should also compare it to the other models I've tried. #LLM#LocalLLaMA#AI#GenerativeAI
does anyone out there know of an in-process vector DB that runs in the same process as the application? like SQLite but for vector stores? #LLM#LLMs#AI#vector#database
Is anyone else in the “trough of disillusionment” part of the Generative AI hype cycle yet? The stuff they output is not good. It takes longer to fix it than writing articles from scratch, IMHO.
Maybe it's a function of caring about the results.
It seems to me that the main problem with #ChatGPT and other #LLMs is context. Each new conversation with them is a clean slate and the longer a conversation goes on the slower and more confused they seem to get. I presume taking the context into account means extra processing time, and storage on their part, but moreover they just don't provide a very good interface for communicating with the #AI about a long-lived project. This is critical for #softwareDevelopment.
An #LLM has access to an enormous amount of data - a dictionary if you will - which makes it absolutely obvious that it can beat existing compression algorithms. If you have access to an enormous dictionary on the decompression side you can achieve much better compression by looking up large chunks of data there instead of storing them.
I’m always baffled by the suggestion that an #LLM could write my articles for me. That only works if plenty of people have written a very similar article already, which begs the question why I should write it in the first place.
A lot of people have responded to my Duolingo post with things like "Never work for free," and "I would never donate my time to a corporation.” Which I completely agree with.
But here's the thing about Duolingo and all of the other companies like it. You already work for them. You just don’t know it.
On Duo, I thought I was learning a language. Participating in the community by helping other learners and building resources seemed like part of the process.
Luis Von Ahn, the CEO of Duolingo, was one of the creators of CAPTCHA, which was originally supposed to stop bot spam by getting a human to do a task a machine couldn’t do. In 2009 Google bought CAPTCHA and used it to get humans to proofread the books they were digitising (without permission from the authors of those books btw). So in order to access much of the web, people had to work for Google. Most of them didn’t know they were working for Google - they thought they were visiting websites.
This is how they get you. They make it seem like they’re giving you something valuable (access to a website, tools to learn a language), while they’re actually taking something from you (your skills, your time, your knowledge, your labour). They make you think they’re helping you, but really you're helping them (and they’re serving you ads while you do it).
Maybe if people had known what CAPTCHA was really for they would’ve done it anyway. Maybe I still would’ve done all that work for Duo if I’d known it would one day disappear from the web and become training data for an LLM ...
... Or maybe I would’ve proofread books for Project Gutenberg, or donated my time to citizen science projects, or worked on an accessibility app, or a million other things which genuinely improve people’s lives and the quality of the web. I didn’t get an informed choice. I got lured into helping a tech company become profitable, while they made the internet a shittier place to be.
How many things are you doing on the web every day which are actually hidden work for tech companies? Probably dozens, or hundreds. We all are. That’s why this is so insidious. It’s everywhere. The tech industry is built on free labour. (And not just free – we often end up paying for the end results of our own work, delivered back to us in garbled, enshittified form).
And it’s a problem that’s only getting worse with AI. Is that thoughtful answer you gave someone on reddit or Mastodon something that will stay on the web for years, helping people in future with the same problem? Or is it just grist for the LLMs?
Back in 2018, Dario Amodei worked at OpenAI. And looking at one of its first A.I. models, he wondered: What would happen as you fed an artificial intelligence more and more data? He and his colleagues decided to study it, and they found that the A.I. didn’t just get better with more data; it got better exponentially.
Does anyone know of an LLM that I can easily run locally on my Linux PC to do simple text cleanup tasks, like removing unnecessary line breaks and the same word twice in a row, capitalize names, etc?
"But first, it’s helpful to think about the purpose the utopian hallucinations about AI are serving. What work are these benevolent stories doing in the culture as we encounter these strange new tools? Here is one hypothesis: they are the powerful and enticing cover stories for what may turn out to be the largest and most consequential theft in human history. Because what we are witnessing is the wealthiest companies in history (Microsoft, Apple, Google, Meta, Amazon …) unilaterally seizing the sum total of human knowledge that exists in digital, scrapable form and walling it off inside proprietary products, many of which will take direct aim at the humans whose lifetime of labor trained the machines without giving permission or consent."
Mark Zuckerberg on Llama 3: Apparently Meta stopped training Llama-3-70b before convergence and decided to move onto Llama-4. Meaning they could have kept training and made it smarter! Also llama3-70b multimodal as well as multilingual and bigger context window are coming. #LLM#AI#MLhttps://youtu.be/bc6uFV9CJGg