And released within hours of each other comes Adam Neely on how hard it will be for #AI to pass a #Music Direction Test https://youtu.be/N8NyEjB_XeA
It is almost like these two coordinated their releases yet they never acknowledge each other's existence
#AI#GenerativeAI#Chatbots#GPT4#Eliza#TuringTest: "The experiment involved 652 participants who completed a total of 1,810 sessions, of which 1,405 games were analyzed after excluding certain scenarios like repeated AI games (leading to the expectation of AI model interactions when other humans weren't online) or personal acquaintance between participants and witnesses, who were sometimes sitting in the same room.
Surprisingly, ELIZA, developed in the mid-1960s by computer scientist Joseph Weizenbaum at MIT, scored relatively well during the study, achieving a success rate of 27 percent. GPT-3.5, depending on the prompt, scored a 14 percent success rate, below ELIZA. GPT-4 achieved a success rate of 41 percent, second only to actual humans.
GPT-3.5, the base model behind the free version of ChatGPT, has been conditioned by OpenAI specifically not to present itself as a human, which may partially account for its poor performance. In a post on X, Princeton computer science professor Arvind Narayanan wrote, "Important context about the 'ChatGPT doesn't pass the Turing test' paper. As always, testing behavior doesn't tell us about capability." In a reply, he continued, "ChatGPT is fine-tuned to have a formal tone, not express opinions, etc, which makes it less humanlike. The authors tried to change this with the prompt, but it has limits. The best way to pretend to be a human chatting is to fine-tune on human chat logs.""
#AI#GenerativeAI#TuringTest#GPT4: "We evaluated GPT-4 in a public online Turing Test. The best-performing GPT-4 prompt passed in 41% of games, outperforming baselines set by ELIZA (27%) and GPT-3.5 (14%), but falling short of chance and the baseline set by human participants (63%). Participants' decisions were based mainly on linguistic style (35%) and socio-emotional traits (27%), supporting the idea that intelligence is not sufficient to pass the Turing Test. Participants' demographics, including education and familiarity with LLMs, did not predict detection rate, suggesting that even those who understand systems deeply and interact with them frequently may be susceptible to deception. Despite known limitations as a test of intelligence, we argue that the Turing Test continues to be relevant as an assessment of naturalistic communication and deception. AI models with the ability to masquerade as humans could have widespread societal consequences, and we analyse the effectiveness of different strategies and criteria for judging humanlikeness."
Facebook has decided to start using all your data to train generative algorithms aka "AI". Here's the form to opt-out. I suggest doing it not only from your personal accounts but any of your corporate accounts as they will be using your IP to train algorithms.
@Abstruse The captcha wanted me to identify the motorbikes in order to proceed, but they are all mopeds and scooters. I think I failed some sort of #TuringTest there.
In conversation with @Jyoti who kinda suggested the idea that AI's could become something like Renaissance patrons, supporting favored artists, and I can't help but think that this is a weird but kinder version of Roko's Basilisk.
Well, at least for the favored artists. Maybe it's still RB for everyone else.
But even if we aren't three ducks in a raincoat, if LLMs can pretend to be three ducks in a raincoat, that'll be enough to convince most people of their sentience, I believe spuriously.
As a kid, I was hardcore #TuringTest on all this and not even Searle's #ChineseRoom could shake me on that. But then I read this:
Imagine if #AlanTuring got in a time machine and got here and everybody's saying "yeah we've got an #AI that can generate text of any length on any topic that sounds convincingly like it was written by a human in seconds, but it's not really intelligent because it doesn't understand what it's writing, so sometimes it writes things that aren't true"
I recognize this is a very long shot, but weirder things have worked out. Does anyone have any photographs from Anchorage, Eagle River, Wasilla, or Palmer, Alaska taken between late 2001 to mid 2002 that might include lamp posts?
I know this is weird as hell, but there is a reason, I swear. I really appreciate the help and the boosts!
@xgranade I so want this to some sort of new Turing test that proves how AI fails at specific tasks, whereas actual people come thru with flying colors (or Alaskan lamp posts from 20 years ago).