My first troublesome hallucination with a #LLM in a while: #Claude3#Opus (200k context) insisting that I can configure my existing #Yubikey#GPG keys to work with PKINIT with #Kerberos and helping me for a couple of hours to try to do so — before realising that GPG keys aren't supported for this use case. Whoops.
No real bother other than some wasted time, but a bit painful and disappointing.
They use #OpenAI, which means my GitHub OSS has almost certainly been used in training data.
They rely on OpenAI's promise to not ingest any code that is used for "context".
They specifically do not disclaim that their tool could result in me violating someone else's copyright, and they could suggest the same code to someone else, too.
Uninstall this crap, now. It's dangerous and irresponsible
Back in 2018, Dario Amodei worked at OpenAI. And looking at one of its first A.I. models, he wondered: What would happen as you fed an artificial intelligence more and more data? He and his colleagues decided to study it, and they found that the A.I. didn’t just get better with more data; it got better exponentially.
I subscribed to Anthropic’s Claude 3 LLM yesterday to get access to their Opus model so I can try it out. It’s £18 per month.
I think LLMs will be useful as research assistants. Any output will need to be fact-checked and heavily edited before publication. People will still be needed who have knowledge of the topic that #LLM is generating text about. But in areas you have existing knowledge, they are a good way to get a high level text to use as a staring point that you can hammer into shape.
The internet is in decay [due to AI]. According to my guest today, Nilay Patel, this isn’t just a blip, as the big platforms figure out how to manage this. He believes that A.I. content will break the internet as we know it.
I've been trialling GitHub Copilot recently at work and, having been generally skeptical of the golden mountains promised by AI hype guys, I have to say that it gave me a modest efficiency gain in some scenarios. I would miss not having it, much like I would miss not having autocomplete.
I'll probably write up a blog for hgrsd.nl with a few thoughts of where it was helpful for me.
i low key don't want to see a big jump in #LLM or #AI capabilities anytime soon. rn they're capable enough that my mom wants to use them, but bad enough that even she has an intuitive sense for when they're wrong
that's how you build "AIQ", the skill of using it. Lots of people toying with them, to feel out their capabilities and limitations
Im as anti-"AI" as the next person, but I think its important to keep in mind the larger strategic picture of "AI" w.r.t. #search when it comes to #DuckDuckGo - both have the problem of inaccurate information, mining the commons, etc. But Google's use of LLMs in search is specifically a bid to cut the rest of the internet out of information retrieval and treat it merely as a source of training data - replacing traditional search with #LLM search. That includes a whole ecosystem of surveillance and enclosure of information systems including assistants, chrome, android, google drive/docs/et al, and other vectors.
DuckDuckGo simply doesnt have the same market position to do that, and their system is set up as just an allegedly privacy preserving proxy. So while I think more new search engines are good and healthy, and LLM search is bad and doesnt work, I think we should keep the bigger picture in mind to avoid being reactionary, and I dont think the mere presence of LLM search is a good reason to stop using it.
VOLlama v0.1.0, an open-source, accessible chat client for OLlama
Unfortunately, many user interfaces for open source large language models are either inaccessible or annoying to use with screen readers, so I decided to make one for myself and others. Non screen reder users are welcome to use it as well.
I hope that ML UI libraries like Streamlit and Gradio will become more friendly with screen readers in the future, so making apps like this is not necessary! #LLM#AI#ML https://chigkim.github.io/VOLlama/
I wanted to understand what I’m missing and get some tips for how I could incorporate A.I. better into my life right now. And Ethan Mollick is the perfect guide: He’s a professor at the Wharton School…who’s spent countless hours experimenting with different chatbots, noting his insights in his newsletter and in a new book, “Co-Intelligence: Living and Working With A.I.”
Yesterday, we looked at how to write a JavaScript app that uses Ollama. Recently, we started to look at Python on this site and I figured that we better follow it up with how to write a Python app that uses Ollama. Just like with JavaScript, Ollama offers a Python library, so we are going to be using that for our examples. Also just like we did with the JavaScript demo, I am going to be using the generate endpoint instead of the chat endpoint. That keeps things simpler but I am going to explore the chat endpoint also at some point.
Install the Ollama Library
The first step is to run pip3 install ollama from the terminal. First, you need to create a virtual environment to isolate your project’s libraries from the global Python libraries.
At this point, we can start writing code. When we used the web service earlier this week, we used the generate endpoint and provided model, prompt, and stream as parameters. We set the stream parameter to false so that it would return a single response object instead of a stream of objects. When using the python library, the stream parameter isn’t necessary because it returns a single response object by default. We still provide it with a model and a prompt, though.
If you run it from the terminal, the response will look familiar.
If you pip install flask to install flask, you can host a simple HTTP page at port 8080 and with the magic of json.loads() and a for loop, you can build your unordered list.
Every time you load the page, it makes a server-side API call to Ollama, gets a list of large cities in Wisconsin, and displays them on the website. The list is never the same (because of hallucinations) but that is another issue.
Have any questions, comments, etc? Please feel free to drop a comment, below.
If you compare the difference between now freely available LLMs like GPT 3.5, Claude Sonnet, LLM3 etc with the paid versions of various models, the difference for most ordinary users/usecases is quite small. That must be a problem for the businesscase of companies like OpenAI, they need large numbers of ordinary people willing to pay every month for access to their models, if you look at the enormous investments? Or are other revenue streams more important ? 🤔 #AI#businessmodels#LLM#GPT
I've had occasion to ask an AI about a thing twice lately (a recent online phenomenon, and a book recommendation). Both times I asked both Gemini and ChatGPT, and both times one gave a reasonable if bland answer, and the other (a different one each time) gave a plausible but completely fictional ("hallucinated") answer.
When do we acknowledge that LLMs, and "AI" in general, aren't quite ready to revolutionize the world?
With all the valid concern around #llm and #genai power and water usage, I thought I'd start a blog series on tiny LLMs. Let's see what they can do on real tasks on very power efficient hardware.
I really like the convention of using ✨ sparkle iconography as an “automagic” motif, e.g. to smart-adjust a photo or to automatically handle some setting. I hate that it has become the defacto iconography for generative AI. 🙁
"The output from an LLM is a derivative work of the data used to train the LLM.
If we fail to recognise this, or are unable to uphold this in law, copyright (and copyleft on which it depends) is dead. Copyright will still be used against us by corporations, but its utility to FOSS to preserve freedom is gone."
>>> Do you happen to know what your context window length is?
Llama: I'm an AI model, and I don't have a fixed "context window" in the classical sense. My training data consists of a massive corpus of text, which I use to generate responses.
We know that the task demands of cognitive tests most scores: if one version of a problem requires more work (e.g., gratuitously verbose or unclear wording, open response rather than multiple choice), people will perform worse.
Incredible research at BlackHat Asia today by Tong Liu and team from the Institute of Information Engineering, Chinese Academy of Sciences (在iie.ac.cn 的电子邮件经过验证)
A dozen+ RCEs on popular LLM framework libraries like LangChain and LlamaIndex - used in lots of chat-assisted apps including GitHub. These guys got a reverse shell in two prompts, and even managed to exploit SetUID for full root on the underlying VM!