"Over the past few months, we’ve learned that Apple has been in discussions with both Google and OpenAI (which owns ChatGPT) about using their respective LLMs to power future features coming to iOS. Now, according to industry analyst Mark Gurman, Apple’s deal with OpenAI might be close to finalized."
Yay, I too got my 7-day suspension badge from Stack Overflow from adding an #LLM#AI disclaimer back after it was first reverted to my four (4) answers!
I've received a formal response by the Stack Overflow team, encouraging me to edit my profile to add any anti-AI statement I want. However, the 7-days ban means that I cannot perform this operation for a week. 🤡
They've also added, and I quote:
While we are working to get further clarity and better public messaging, it is our current understanding (based on this official statement from Stack Exchange) that these agreements with AI companies are a way to work with companies using Stack Exchange data to provide appropriate attribution (as per the CC by-SA license), rather than simply providing answers without attribution, as many do today.
I may have lost my temper and delivered them this response to this paragraph:
And I'm glad you unpaid clowns drank the AI Kool-Aid, it makes easier my decision to put Stack Overflow to the overflowing enshittification bin where it just rightfully earned its place.The sad truth is that LLM systems cannot provide appropriate attribution by design. These systems are meant to generate plausible looking texts with unreliable accuracy. This includes attribution that will also be plausible-looking once it goes through the data mixer.
Enjoy your joyless reverts and bans while you can, shills.
A study that confirms what I’ve been suspecting for a while: fine-tuning a #LLM with new knowledge increases its tendency to hallucinate.
If the new knowledge wasn’t provided in the original training set, then the model has to shift its weights from their previous optimal state to a new state that has to accommodate both the previous and new knowledge - and it may not necessarily be optimal.
Without a new validation round against the whole previous cross-validation and test sets, that’s just likely to increase the chances for the model to go off the tangent.
@m0bi13 BTW, najlepsze modele LLM mają około 50% skuteczności w średnich zadaniach matematycznych… proste w około 80% a praktycznie 0 w trudnych ;)
To że chatGPT potrafi liczyć to po prostu wewnątrz są tzw function-calling… czyli model wie że jak zobaczy zadanie matematyczne to ma je wywxtraktowac i poprosić komputer o wykonanie tego zadania.
So, how can we get a proper answer? Ten years ago, when I wrote “The Milwaukee Soup App”, I used the Kimono (which is long dead) to scrape the soup of the day. You could also write a fiddly script to scrape the value manually. It turns out that there is another option, though. You could use Scrapegraph-ai. ScrapeGraphAI is a web scraping Python library that uses LLM and direct graph logic to create scraping pipelines for websites, documents, and XML files. Just say which information you want to extract and the library will do it for you.
Let’s take a look at an example. The project has an official demo where you need to provide an OpenAI API key, select a model, provide a link to scrape, and write a prompt.
As you can see, it reliably gives you the flavor of the day (in a nice JSON object). It will go even further, though because if you point it at the monthly calendar, you can ask it for the flavor of the day and soup of the day for the remainder of the month and it can do that as well.
I am running Python 3.12 on my Mac but when you run pip install scrapegraphai to install the dependencies, it throws an error. The project lists the prerequisite of Python 3.8+, so I downloaded 3.9 and installed the library into a new virtual environment.
Let’s see what the code looks like.
You will notice that just like in yesterday’s How to build a RAG system post, we are using both a main model and an embedding model.
At this point, if you want to harvest flavors of the day for each location, you can do so pretty simply. You just need to loop through each of Culver’s location websites.
Have a question, comment, etc? Please feel free to drop a comment, below.
Just got an e-mail from #storyblok informing me that one of their sub-processors (whatever that means) is now #OpenAI.
I immediately asked them if the content I created will be used in any way to train their #LLM and if so, I want no part in it. Requested a back-up of my data and to delete everything.
I used it for the first iteration of my blog but switched to #markdown a while ago.
Stack Overflow, a popular forum for programmers and software developers, announced a partnership with OpenAI earlier this week, selling the site’s data, including users’ forum posts, to train ChatGPT.
Now unhappy users are finding themselves banned for editing their popular posts in protest, and even finding those posts changed back by admin – “a reminder that anything you post on any of these platforms can and will be used for profit,” concluded one. Futurism has more.
Not likely in the US. I personally believe that it's unlikely any of the copyright claims against AI generators will stand.
If Google Books -a web site that literally copied full copyrighted works and allowed people to search through them and showed them actual images of the search results- was deemed to not be in violation of copyright law, then I just don't see how AI systems are going to be found in violation.
Ultimately, if just analyzing and learning from prior works and using that knowledge to create derivative works is copyright violation, then literally every single human that has ever drawn or written anything at all is also in violation.
I just found out one actually useful use case for #LLM (a.k.a. “AI”).
There is thousands of tonnes of documents from #german#nazi#regime detailing basically everything what happened, who was involved and who participated what and said what.
Feed ALL of that to an LLM and ask it “how can we avoid this happening ever again”. I have a pretty good guess my self but some people aren’t that convinced. Perhaps this is The Key. #history#sciense#neverAgain
(1/2) Prompt Fuzzer - a new open-source project for LLM security 👇🏼
Prompt Fuzzer is a new open-source project that provides a set of functions for assessing the security of GenAI applications. This CLI-based tool enables you to run and test your system prompts to identify security vulnerabilities against potential dynamic LLM-based attacks.
(2/2) The CLI interface has an interactive playground chat interface, giving you the chance to iteratively improve your system prompt, hardening it against a wide spectrum of generative AI attacks
Back in January, we started looking at AI and how to run a large language model (LLM) locally (instead of just using something like ChatGPT or Gemini). A tool like Ollama is great for building a system that uses AI without dependence on OpenAI. Today, we will look at creating a Retrieval-augmented generation (RAG) application, using Python, LangChain, Chroma DB, and Ollama. Retrieval-augmented generation is the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response. If you have a source of truth that isn’t in the training data, it is a good way to get the model to know about it. Let’s get started!
Your RAG will need a model (like llama3 or mistral), an embedding model (like mxbai-embed-large), and a vector database. The vector database contains relevant documentation to help the model answer specific questions better. For this demo, our vector database is going to be Chroma DB. You will need to “chunk” the text you are feeding into the database. Let’s start there.
Chunking
There are many ways of choosing the right chunk size and overlap but for this demo, I am just going to use a chunk size of 7500 characters and an overlap of 100 characters. I am also going to use LangChain‘s CharacterTextSplitter to do the chunking. It means that the last 100 characters in the value will be duplicated in the next database record.
The Vector Database
A vector database is a type of database designed to store, manage, and manipulate vector embeddings. Vector embeddings are representations of data (such as text, images, or sounds) in a high-dimensional space, where each data item is represented as a dense vector of real numbers. When you query a vector database, your query is transformed into a vector of real numbers. The database then uses this vector to perform similarity searches.
You can think of it as being like a two-dimensional chart with points on it. One of those points is your query. The rest are your database records. What are the points that are closest to the query point?
Our main model for this demo is going to be phi3. It is a 3.8B parameters model that was trained by Microsoft.
LangChain
You will notice that today’s demo is heavily using LangChain. LangChain is an open-source framework designed for developing applications that use LLMs. It provides tools and structures that enhance the customization, accuracy, and relevance of the outputs produced by these models. Developers can leverage LangChain to create new prompt chains or modify existing ones. LangChain pretty much has APIs for everything that we need to do in this app.
The Actual App
Before we start, you are going to want to pip install tiktoken langchain langchain-community langchain-core. You are also going to want to ollama pull phi3 and ollama pull nomic-embed-text. This is going to be a CLI app. You can run it from the terminal like python3 app.py "<Question Here>".
You also need a sources.txt file containing the URLs of things that you want to have in your vector database.
The May 7th event is too recent to be in the model’s training data. This makes sure that the model knows about it. You could also feed the model company policy documents, the rules to a board game, or your diary and it will magically know that information. Since you are running the model in Ollama, there is no risk of that information getting out, too. It is pretty awesome.
Have any questions, comments, etc? Feel free to drop a comment, below.