I just came across this (h/t to Peter Krupa), and it blew my mind. It highlights the problem with LLMs in general with pinpoint accuracy, and wraps it in a well known metaphorical idiom that everyone understands — which instantly becomes a meta reference. …
@lars i wish there were a psychologist around to offer their thoughts, but if you asked me to imagine a room without an elephant, i’d picture the same thing, and i’d have to try hard to remove the elephant from the image. i suspect that we share a similar limited aspect of pattern matching intelligence, but we have several more powerful analytical systems that kick in. kinda like system 1 vs system 2 in Thinking Fast and Slow
@kellogh I can't reproduce the response now — presumably they've "plugged" it after it went viral. But here's the Dall-E prompt ChatGPT produces when I try:
"An empty room featuring a minimalist design. The room has plain white walls, a light wooden floor, and a large window with sheer white curtains, allowing soft natural light to fill the space. There are no furnishings or decorations, and notably, there is no elephant in the room, emphasizing the emptiness and simplicity of the space."
@lars this whole thing is super interesting. It sounds like the strange behavior is more in diffusion models (Dall-E) than language models (GPT4). it reminds me of that paper that found that LLMs only learn logic in a single direction, e.g. if none of its training data had “elephant not in a room”, it wouldn’t know what to do and instead pattern match to “elephant in room”
Add comment