I wanted to understand what I’m missing and get some tips for how I could incorporate A.I. better into my life right now. And Ethan Mollick is the perfect guide: He’s a professor at the Wharton School…who’s spent countless hours experimenting with different chatbots, noting his insights in his newsletter and in a new book, “Co-Intelligence: Living and Working With A.I.”
Yesterday, we looked at how to write a JavaScript app that uses Ollama. Recently, we started to look at Python on this site and I figured that we better follow it up with how to write a Python app that uses Ollama. Just like with JavaScript, Ollama offers a Python library, so we are going to be using that for our examples. Also just like we did with the JavaScript demo, I am going to be using the generate endpoint instead of the chat endpoint. That keeps things simpler but I am going to explore the chat endpoint also at some point.
Install the Ollama Library
The first step is to run pip3 install ollama from the terminal. First, you need to create a virtual environment to isolate your project’s libraries from the global Python libraries.
At this point, we can start writing code. When we used the web service earlier this week, we used the generate endpoint and provided model, prompt, and stream as parameters. We set the stream parameter to false so that it would return a single response object instead of a stream of objects. When using the python library, the stream parameter isn’t necessary because it returns a single response object by default. We still provide it with a model and a prompt, though.
If you run it from the terminal, the response will look familiar.
If you pip install flask to install flask, you can host a simple HTTP page at port 8080 and with the magic of json.loads() and a for loop, you can build your unordered list.
Every time you load the page, it makes a server-side API call to Ollama, gets a list of large cities in Wisconsin, and displays them on the website. The list is never the same (because of hallucinations) but that is another issue.
Have any questions, comments, etc? Please feel free to drop a comment, below.
If you compare the difference between now freely available LLMs like GPT 3.5, Claude Sonnet, LLM3 etc with the paid versions of various models, the difference for most ordinary users/usecases is quite small. That must be a problem for the businesscase of companies like OpenAI, they need large numbers of ordinary people willing to pay every month for access to their models, if you look at the enormous investments? Or are other revenue streams more important ? 🤔 #AI#businessmodels#LLM#GPT
I've had occasion to ask an AI about a thing twice lately (a recent online phenomenon, and a book recommendation). Both times I asked both Gemini and ChatGPT, and both times one gave a reasonable if bland answer, and the other (a different one each time) gave a plausible but completely fictional ("hallucinated") answer.
When do we acknowledge that LLMs, and "AI" in general, aren't quite ready to revolutionize the world?
With all the valid concern around #llm and #genai power and water usage, I thought I'd start a blog series on tiny LLMs. Let's see what they can do on real tasks on very power efficient hardware.
I really like the convention of using ✨ sparkle iconography as an “automagic” motif, e.g. to smart-adjust a photo or to automatically handle some setting. I hate that it has become the defacto iconography for generative AI. 🙁
"The output from an LLM is a derivative work of the data used to train the LLM.
If we fail to recognise this, or are unable to uphold this in law, copyright (and copyleft on which it depends) is dead. Copyright will still be used against us by corporations, but its utility to FOSS to preserve freedom is gone."
>>> Do you happen to know what your context window length is?
Llama: I'm an AI model, and I don't have a fixed "context window" in the classical sense. My training data consists of a massive corpus of text, which I use to generate responses.
We know that the task demands of cognitive tests most scores: if one version of a problem requires more work (e.g., gratuitously verbose or unclear wording, open response rather than multiple choice), people will perform worse.
That installs Ollama as a dependency in package.json.
Basic CLI example
At this point, we can start writing code. When we used the web service earlier this week, we used the generate endpoint and provided model, prompt, and stream as parameters. We set the stream parameter to false so that it would return a single response object instead of a stream of objects. When using the javascript library, the stream parameter isn’t necessary because it returns a single response object by default. We still provide it with a model and a prompt, though.
If you run it from the terminal, the response will look familiar.
If you npm install express to install express, you can host a simple HTTP page at port 8080 and with the magic of JSON.parse() and a for loop, you can build your unordered list.
Every time you load the page, it makes a server-side API call to Ollama, gets a list of large cities in Wisconsin, and displays them on the website. The list is never the same (because of hallucinations) but that is another issue.
Have any questions, comments, etc? Please feel free to drop a comment, below.
Ce matin... deux BOT de scrapping pour alimenter des modèles d'IA/#LLM ont abusé du forum d'@osm_fr
C'est pas la première fois et ça devient vraiment une plaie, surtout quand #ClaudeBot requête les URL de notre ancien #phpBB, remplacé il y a plusieurs années par #discourse
Malgrès plus de 130 000 erreurs 404 rien que ce matin, il continuait à un rythme effréné...
Autre bot albert-bot... de albertai.com (rien avoir avec l'Albert cocorico), bloqué lui aussi.
Morgen nicht verpassen: Die Tagung "No risk, no innovation? Künstliche Intelligenz in der Museumspraxis" beschäftigt sich mit KI-basierten Technologien im Museumsbereich. Auch am #LMWStuttgart setzen wir uns mit den Einsatzmöglichkeiten von KI-Technologien am Museum auseinander. Den Grundstein dafür legen unsere KI-Ethikrichtlinien: https://github.com/LMWStuttgart/KI-Ethik
Asked LLama-3 to implement a CRC32 routine in C. The 8B model.
With the exception of it forgetting to declare the table array, the code compiled without errors.
I also asked it to run the code on a test string, which it did and explained at each step what the intermediate CRC32 was.
Well. The result was wrong. Both when it executed the code itself, as well as when I compiled and ran it ;)
But this would definitely confuse someone who tried to use it for coding. I see nothing wrong with the code - it all looks perfect. If I get the time I might look into why it's not correct.
Very nice picture that was shared by Ronald van Loon on X, you can discuss if the categories are complete and correct, but it illustrates that the field of AI is much more then just transformers/LLMs. #AI#Machinelearning#neuralnetworks#deeplearning#LLM#Transfomers
My favorite client for MacOS is MindMac. You can buy it for under $30, it works with multiple models, servers, and server types, and it is easy to use.
If you want to look further into it, you can check it out at mindmac.app.
Android
My favorite client for Android is Amallo. It is $23 and like MindMac, it works with multiple models, servers, and server types. My only complaint would be that uploading a base64-encoded image to the model doesn’t seem to work well.
Incredible research at BlackHat Asia today by Tong Liu and team from the Institute of Information Engineering, Chinese Academy of Sciences (在iie.ac.cn 的电子邮件经过验证)
A dozen+ RCEs on popular LLM framework libraries like LangChain and LlamaIndex - used in lots of chat-assisted apps including GitHub. These guys got a reverse shell in two prompts, and even managed to exploit SetUID for full root on the underlying VM!