Asked #Copilot (formerly #BingChat) a familiar riddle but with numbers changed to make it impossible. It generated the same solution but substituting the numbers so that it ends up with the nonsense claim:
"I have an empty opaque bag. I put two apples and one banana in the bag. I either remove the banana or I remove one apple. I then remove all remaining fruits from the bag. Is it possible to tell what is in the bag now?"
@bornach I was going to post something like "I guess programmers' jobs are safe" but as I was looking at it I realized that for most companies, 15 is close enough to 23 that they'll just use the AI and call it a win
Or maybe it was taught by BingChat/Copilot - isn't Microsoft reportedly using GPT-4? Its solution is even more tortuous and later admits to measuring the wrong amount of water. It never realises it could have stopped after step 1.