I'll admit I find #LLMs fascinating, although not for the reason so many others seem to.
I'm not impressed by their answers -- you can quickly see how flaky they are. I'm impressed by their apparent ability to understand the question.
That word "apparent" is load-bearing. They can't, of course. Can't reason or understand. And yet this apparent ability is strong enough that you can instruct them in plain English to subvert their own design.
