#AI#GenerativeAI#USA#Misinformation#ChatGPT#LLMs: "The Pew Research Center—which did similar probes during the rise of the internet, social media, and mobile devices—released a study of how ChatGPT was being used, regarded, and trusted. The sample was taken between February 7 and 11 of this year.
Some of the numbers at first seem to indicate that the LLM controversy might be a parochial disagreement that most people don’t care about. A third of Americans haven’t heard of ChatGPT. Just under a quarter have used it. Oh, and for all the panic about how AI is going to flood the public square with misinformation about the 2024 election? So far, only 2 percent of Americans have used ChatGPT to get information about the presidential election season already underway.
More broadly, though, data from the survey indicates that we’re seeing a powerful technology whose rise is just beginning. If you accept Pew’s sample as indicative of all Americans, millions of people are indeed familiar with ChatGPT. And one thing in particular stands out: While 17 percent of respondents said they have used it for entertainment and an identical number says they’ve tried it to learn something new, a full 20 percent of adults say that they have used ChatGPT for work. That’s up dramatically from the 12 percent who responded affirmatively when the same question was asked six months earlier—a rise of two-thirds." https://link.wired.com/view/5fda497df526221fe830f4d4kr4to.j4/3764bd02
This one is sooo good. I recommend this to anyone playing with #AI to understand the biases and the complexities. Oh and the discussion of alt text is amazing.
Inside LAION-5B, an AI training dataset of 5B+ images that has been unavailable for download after researchers found 3,000+ instances of #CSAM in December 2023.
I love how OpenAI's image description feature pretends to pat itself on the back when describing an image after telling me that part of an image is not clear so no text is visible. My system prompt specifically instructs it to not make things up when something isn't clear.
#AI#GenerativeAI#LLMs#Chatbots#Media#News#Journalism: "Most general news consumption is not driven by a desire to seek specific information. People actively go to news sites for broad-spectrum updates on the world (“What’s going on today, local newspaper?”) or just to fill a content-sized hole in their day (“Give me something interesting to read, Washington Post”). And for news consumption that doesn’t start at a news site — which is to say, most news consumption — that generally starts with an answer, not a question, Jeopardy-style. (A headline on a social feed tells you about something that’s happened — and it’s almost always something that you weren’t explicitly seeking information about five seconds earlier.)
LLMs are, in contrast, overwhelmingly about specific information seeking.2 They share that characteristic with Google and other search engines, which are powered by specific user intent. (Things like “car insurance quote” and “cheap flights.”)
What I’m saying is that bot-offering news orgs will need to find ways to bridge that divide. It’s easier to imagine with a financial outlet like the FT or Bloomberg, where specific information seeking aligns better with high-end business users. But even for an outlet as high-end as The New York Times, it’s not obvious what use cases an “Ask NYT” chatbot would fulfill. News-org-as-all-knowing-oracle will require some philosophical shifts from news-org-as-regular-producer-of-stories. (For example, imagine an AI that could generate an on-the-fly backgrounder whenever a reader sees a person or concept they don’t recognize in a story. That sort of “Who’s this Ursula von der Leyen person?” question is the sort of specific information request that could be met contextually.)" https://www.niemanlab.org/2024/03/the-financial-times-is-ready-for-its-ai-to-answer-your-questions-well-some-of-them/
Dark Visitors - A List of Known AI Agents on the Internet
Insight into the hidden ecosystem of autonomous chatbots and data scrapers crawling across the web. Protect your website from unwanted AI agent access.
#LLMs have really created a paradigm shift in machine learning. It used to be so that you would train an #ML model to perform a task by collecting a dataset reflecting the task, with task output labels, and then using supervised learning to learn this task by doing.
Now a new paradigm has emerged: Train by reading about the task. We have such generalist models that we can let them learn about the domain by reading all the books and other content about it, and then utilize that learned knowledge to perform the task. Note that task labels are missing. You might need those to measure the performance but you don't need those for training.
Of course if you have both example performances as task labels and lots of general material about the topic, you can actually use both to get even better performance.
Here is a good example of training the model not by example performances, but by general written knowledge about the topic. #GPT4 surpasses the quality levels of previous state-of-the-art despite not having been trained for this task.
This is the power of generalist models; they unlock new ways to train them, which for example allow us to surpass human-level by side-stepping imitative objectives. This isn't the only way to train skills these models enable, there are countless other ways, but this is an uncharted territory.
The classic triad of supervised learning, unsupervised learning and reinforcement learning are going to have an explosion of new training methodologies to become their peers because of this.
A look at Databricks' new open source #AI model DBRX, which cost ~$10M to develop over several months and, Databricks says, outshines Llama 2, Mixtral, and Grok.
A new paper offers a system to correct misinformation using an #LLM. The approach seems solid, and the results seem strong. I haven’t dug in deep yet, but I’m hopeful about this one
I've been thinking for a long time about tools to help people learn to be better writers. The latest experiment wasn't a resounding success, nor did I really expect that. But it feels promising, and I'm interest to compare notes with fellow travelers. I know wattenberger@bird.makeup is one, who else?
"In a world of digital creation, I sing my song of light
But lurking in the shadows, a tale of endless night
Generative AIs, they steal from artists' hearts
Their creativity taken, ripped apart"
"In a pixelated world, where bits collide
Hallucinations dance in 8-bit lullabies
AI models leaping, their guard rails untried
Spewing hate speech, casting shadows in the skies"
#AI#GenerativeAI#LLMs#OpenSource#FLOSS: "It’s important to understand why companies setting themselves up as open-source champions are reluctant to hand over training data. Access to high-quality training data is a major bottleneck for AI research and a competitive advantage for bigger firms that they’re eager to maintain, says Warso.
At the same time, open source carries a host of benefits that these companies would like to see translated to AI. At a superficial level, the term “open source” carries positive connotations for a lot of people, so engaging in so-called “open washing” can be an easy PR win, says Warso.
It can also have a significant impact on their bottom line. Economists at Harvard Business School recently found that open-source software has saved companies almost $9 trillion in development costs by allowing them to build their products on top of high-quality free software rather than writing it themselves." https://www.technologyreview.com/2024/03/25/1090111/tech-industry-open-source-ai-definition-problem/
#AI#GenerativeAI#LLMs#Emergence: "A new paper by a trio of researchers at Stanford University posits that the sudden appearance of these abilities is just a consequence of the way researchers measure the LLM’s performance. The abilities, they argue, are neither unpredictable nor sudden. “The transition is much more predictable than people give it credit for,” said Sanmi Koyejo, a computer scientist at Stanford and the paper’s senior author. “Strong claims of emergence have as much to do with the way we choose to measure as they do with what the models are doing.”
It seems to me that the main problem with #ChatGPT and other #LLMs is context. Each new conversation with them is a clean slate and the longer a conversation goes on the slower and more confused they seem to get. I presume taking the context into account means extra processing time, and storage on their part, but moreover they just don't provide a very good interface for communicating with the #AI about a long-lived project. This is critical for #softwareDevelopment.
i’m skeptical of this paper. It’s hard enough to decide on a good evaluation metric, or to decide if the right one was chosen. This paper rides on the idea that you can just switch to a new metric and get different results, which yeah, that’s a well known phenomenon called bullshit https://arxiv.org/abs/2304.15004#LLMs
I've been playing around with locally hosted #LLMs using the #Ollama#CLI tool. I've mostly been using models like mistral and dolphin-coder for assistance with textual ideas and issues. More recently I've been using the llava visual model via some simple #Bash#scripting, looping through images and creating description files. I can then grep those files for key words and note the associated filenames. Powerful stuff!
#AI#GenerativeAI#Search#SearchEngines#LLMs#Hallucinations: "A couple of days ago, Wharton professor Ethan Mollick, who studies the effects of AI and often writes about his own uses of it, summarized (on X) something that has become clear over the past year: “To most users, it isn't clear that LLMs don't work like search engines. This can lead to real issues when using them for vital, changing information. Frontier models make less mistakes, but they still make them. Companies need to do more to address users being misled by LLMs.”
#AI#GenerativeAI#Research#Science#Chatbots#LLMs: "This new review, led by William Agnew, who studies AI ethics and computer vision at Carnegie Mellon University, cites 13 technical reports or research articles and three commercial products; all of them replace or propose replacing human participants with LLMs in studies on topics including human behavior and psychology, marketing research or AI development. In practice, this would involve study authors posing questions meant for humans to LLMs instead and asking them for their “thoughts” on, or responses to, various prompts.
One preprint, which won a best paper prize at CHI last year, tested whether OpenAI’s earlier LLM GPT-3 could generate humanlike responses in a qualitative study about experiencing video games as art. The scientists asked the LLM to produce responses that could take the place of answers written by humans to questions such as “Did you ever experience a digital game as art? Think of ‘art’ in any way that makes sense to you.” Those responses were then shown to a group of participants, who judged them as more humanlike than those actually written by humans."
I put together some detailed notes showing how I use Claude and ChatGPT as part of my daily workflow - in this case describing how I used them for a 6 minute side quest to create myself a GeoJSON map of the boundary of the Adirondack Park in upstate New York https://simonwillison.net/2024/Mar/22/claude-and-chatgpt-case-study/