Neu ist diese Info nicht und doch wird es weiträumig KI bei Programmierung eingesetzt. Deswegen aber auch Zeit für die Korrektur vom Code verschwendet.
»Qualitätsmängel – Forscher warnen vor ChatGPT-Einsatz beim Programmieren:
Viele Programmierer:innen greifen bei der Softwareentwicklung zu KI-Chatbots wie ChatGPT. Das sollten sie aber eher bleiben lassen, wie jetzt eine Studie nahelegt.«
Modern #AI text generators create randomized output with no prior planning. They resist to be quality-checked by tools and processes established in the software industry.
Given this, the results are amazing. However, companies are selling the idea that these assistants will do quality checking themselves soon™.
This is mass delusion. But hey, the perks for managers/investors are worthwhile 🤷.
Measuring correctness is very hard, so any percentage, especially a high one, needs context:
What problems do you pose to the #AI?
How is the accuracy of the answer measured?
How is reproducibility of the measurement ensured? (Also over small modifications of the training data or code? I.e. regression tests?)
Right now, I can be sure that my hammer hits the nail, unless my aim is off or the hammerhead was already loose. AI tools don't have that property and maybe never will. @TheServitor
@marcel@TheServitor correctness is a little wonky to talk about, because the comparison isn’t direct. in engineering, we assume that 100% correctness isn’t achievable, if you haven’t found bugs, you just haven’t used it long enough. but with a program, every time you find a bug and fix it, you’re permanently closer to correct, whereas with an LLM, each inference is a new case. all you can do is optimize the entire model. so LLM correctness is statistically extremely difficult
Related - if anyone is looking for a web dev with agency experience working on modern headless + serverless stacks, toot in my general direction. #chatgpt#ai#webdev
Hello Mastodon, I know that a lot of you discuss the high environmental cost (such as energy use and water use) of AI and I hope that some of you could reply with authoritative publications/links regarding this problem! I want to try to convince an environmental science colleague #climatechange#AI#chatgpt#energy#technology#machinelearning#llm