"There has been a shift in the #AI space: some models, like #ChatGPT & #Gemini, have evolved into entire web platforms spanning multiple use cases & access points. Other large language models like #LLaMa or #OLMo, though technically speaking they share a basic architecture, don’t actually fill the same role. They are intended to live in the background as a service or component, not in the foreground as a name brand." https://techcrunch.com/2024/04/19/too-many-models/
Yesterday, we looked at how to write a JavaScript app that uses Ollama. Recently, we started to look at Python on this site and I figured that we better follow it up with how to write a Python app that uses Ollama. Just like with JavaScript, Ollama offers a Python library, so we are going to be using that for our examples. Also just like we did with the JavaScript demo, I am going to be using the generate endpoint instead of the chat endpoint. That keeps things simpler but I am going to explore the chat endpoint also at some point.
Install the Ollama Library
The first step is to run pip3 install ollama from the terminal. First, you need to create a virtual environment to isolate your project’s libraries from the global Python libraries.
At this point, we can start writing code. When we used the web service earlier this week, we used the generate endpoint and provided model, prompt, and stream as parameters. We set the stream parameter to false so that it would return a single response object instead of a stream of objects. When using the python library, the stream parameter isn’t necessary because it returns a single response object by default. We still provide it with a model and a prompt, though.
If you run it from the terminal, the response will look familiar.
If you pip install flask to install flask, you can host a simple HTTP page at port 8080 and with the magic of json.loads() and a for loop, you can build your unordered list.
Every time you load the page, it makes a server-side API call to Ollama, gets a list of large cities in Wisconsin, and displays them on the website. The list is never the same (because of hallucinations) but that is another issue.
Have any questions, comments, etc? Please feel free to drop a comment, below.
Someone #AI used to depict a school principal making racist comments about students, prompting a flood of angry messages and the principal's temporary removal.
We are not prepared for the world we have created.
One of our users has generated a song about Organic Maps. How does it sound to you? 😀
Beneath the twinkling stars, I stroll hand in hand,
With Organic Maps, exploring enchanted lands.
From Eiffel's gentle glow to Iceland's black sand shore,
Every step a melody, every sight, pure amore.
5 Mins Read
The Bezos Earth Fund has announced an AI for Climate and Nature Grand Challenge, promising up to $100M in grants for solutions leveraging artificial intelligence – and alternative proteins are one of its key focus areas
If you compare the difference between now freely available LLMs like GPT 3.5, Claude Sonnet, LLM3 etc with the paid versions of various models, the difference for most ordinary users/usecases is quite small. That must be a problem for the businesscase of companies like OpenAI, they need large numbers of ordinary people willing to pay every month for access to their models, if you look at the enormous investments? Or are other revenue streams more important ? 🤔 #AI#businessmodels#LLM#GPT
Kinda similar to large quantities of "#AI howtos" that show you how to use certain libraries or pipelines. When trying to reproduce the results you often find that the way the howto is structured, things don't actually work, that the claimed results don't make sense or that the library just doesn't work the way the author (which is probably an "AI") claims.
Because so much of the reception of these papers (especially here) is just "WOW, the future is here", "this is so cool" or (the literal worst): "The changes everything".
The reproducebility crisis in psychology is a joke compared to a lot of AI stuff.
Sidenote: For bigger models there's only a handful of companies on the planet who have the resources (as in NVIDIA graphics cards and limitless cash to burn on energy and data) to build and test these models.
I've had occasion to ask an AI about a thing twice lately (a recent online phenomenon, and a book recommendation). Both times I asked both Gemini and ChatGPT, and both times one gave a reasonable if bland answer, and the other (a different one each time) gave a plausible but completely fictional ("hallucinated") answer.
When do we acknowledge that LLMs, and "AI" in general, aren't quite ready to revolutionize the world?
There's a funny thing you see in many scientific papers - especially #AI papers: The paper will prominently include a link to a GitHub repository with claims of code availability "soon" but when you go there (months after the paper was released) there's either just a placeholder or the paper text.
People use GitHub links to score browny points for "doing open science" but most of it is just not there. Especially with statistical systems when you realize that you don't get the training data, you don't get the code, you don't get model weights what you get are results and a "trust me bro".
Incredible research at BlackHat Asia today by Tong Liu and team from the Institute of Information Engineering, Chinese Academy of Sciences (在iie.ac.cn 的电子邮件经过验证)
A dozen+ RCEs on popular LLM framework libraries like LangChain and LlamaIndex - used in lots of chat-assisted apps including GitHub. These guys got a reverse shell in two prompts, and even managed to exploit SetUID for full root on the underlying VM!