I just issued a data deletion request to #StackOverflow to erase all of the associations between my name and the questions, answers and comments I have on the platform.
One of the key ways in which #RAG works to supplement #LLMs is based on proven associations. Higher ranked Stack Overflow members' answers will carry more weight in any #LLM that is produced.
By asking for my name to be disassociated from the textual data, it removes a semantic relationship that is helpful for determining which tokens of text to use in an #LLM.
If you sell out your user base without consultation, expect a backlash.
The TinyChart-3B LLM answers questions about data visualizations. It can also generate underlying data from a dataviz and Python code to re-create a similar chart.
I've had occasion to ask an AI about a thing twice lately (a recent online phenomenon, and a book recommendation). Both times I asked both Gemini and ChatGPT, and both times one gave a reasonable if bland answer, and the other (a different one each time) gave a plausible but completely fictional ("hallucinated") answer.
When do we acknowledge that LLMs, and "AI" in general, aren't quite ready to revolutionize the world?
I was listing something on eBay, and they encourage starting with an existing listing—presumably to increase the amount of detail and decrease the amount of work.
When I selected the same model, I got a default description that was extremely robotic and wordy while just repeating the spec sheet. I thought it sounded LLM-generated; sure enough when I went to edit it, there is a big shiny “write with AI” button.
"The biggest question raised by a future populated by unexceptional A.I., however, is existential. Should we as a society be investing tens of billions of dollars, our precious electricity that could be used toward moving away from fossil fuels, and a generation of the brightest math and science minds on incremental improvements in mediocre email writing?" (From an NYT article. See original thread.)
I just tried a few AI plugins for #figma and they were all bad. This domain might be a great test for #LLMs . I predict these failings are unlikely to be fixed any time soon:
Layout was poor
They can't create components
Laughably complex object hierarchies (everything was enclosed in a frame)
Of course things will improve, but I expect fixing these deep structural problems are a function of many new constraints, likely beyond what today's LLMs are actually capable of. @simon ?
Saying "LLMs will eventually do every job" is a bit like:
Seeing Wifi wireless data
Then predicting "Wireless" Power saws (no electrical cord or battery) are just around the corner
It's a misapplication of the tech. You need to understand how #LLMs work and extrapolate that capability. It's all text people. Summarizing, collating, template matching. All fair game. But stray outside of that box and things get much harder.
alright, i have to declare this as a strong opinion — #LLMs are better at alt-text than people are
the goal of alt text is to let a person “without eyes” see the picture, to get the same experience as someone who can see fine
but often, almost always, human-written alt text is either too succinct to be helpful, or just an extension of the post itself, and so doesn’t help an impaired person understand what’s in it
Do you REALLY want to get a feel for how GPT-4o does what it does? Just complete this poem — by doing so, you’ll have performed a computation similar to the one it does when you feed it a text-plus-image prompt.
i used an analogy yesterday, that #LLMs are basically system 1 (from Thinking Fast and Slow), and system 2 doesn’t exist but we can kinda fake it by forcing the LLM to have an internal dialog.
my understanding is that system 1 was more tuned to pattern matching and “gut reactions”, while system 2 is more analytical
i think it probably works pretty well, but curious what others think
“The general problem of mixing data with commands is at the root of many of our computer security vulnerabilities.” Great explainer by security researcher Bruce Schneier on why large language models may not be a great choice for tasks like processing your emails. https://cacm.acm.org/opinion/llms-data-control-path-insecurity/
i generally regard, “i will think less of you” type comments as a joke, because of how ridiculous the sentiment is, but this sort of stuff is perverse on the fedi
"When the Singaporean government asked local writers if they would agree to having their work used to train a large language model, it probably did not expect the country’s tiny literary community to react so fiercely."
i've been getting into the things #LLMscan't do well, because i think it says a lot about what they're useful for, and it helps build a mental model around how they work
Eventually, people may stop writing, stop filming, stop composing—at least for the open, public web. People will still create, but for small, select audiences, walled-off from the content-hoovering AIs.
If we continue in this direction, the #web—that extraordinary ecosystem of knowledge production—will cease to exist in any useful form.