Why AI is going to be a shitshow.

I was just watching a tiktok with a black girl going over how race is a social construct. This felt wrong to me so I decided to back check her facts.

(she was right, BTW)

Now I’ve been using Microsoft’s Copilot which is baked into Bing right now. It’s fairly robust and sure it has it’s quirks but by and large it cuts out the middle man of having to find facts on your own and gives a breakdown of whatever your looking for followed by a list of sources it got it’s information from.

So I asked it a simple straightforward question:

“I need a breakdown on the theory behind human race classifications”

And it started to do so. quite well in fact. it started listing historical context behind the question and was just bringing up Johann Friedrich Blumenbach, who was a German physician, naturalist, physiologist, and anthropologist. He is considered to be a main founder of zoology and anthropology as comparative, scientific disciplines. He has been called the “founder of racial classifications.”

But right in the middle of the breakdown on him all the previous information disappeared and said, I’m sorry I can’t provide you with this information at this time.

I pointed out that it was doing so and quite well.

It said that no it did not provide any information on said subject and we should perhaps look at another subject.

Now nothing i did could have fallen under some sort of racist context. i was looking for historical scientific information. But Bing in it’s infinite wisdom felt the subject was too touchy and will not even broach the subject.

When other’s, be it corporations or people start to decide which information a person can and cannot access, is a damn slippery slope we better level out before AI starts to roll out en masse.

PS. Google had no trouble giving me the information when i requested it. i just had to look up his name on my own.

ElectroVagrant,

When other’s, be it corporations or people start to decide which information a person can and cannot access, is a damn slippery slope we better level out before AI starts to roll out en masse.

You highlight the bigger issue here than AI alone tbh. This is why another critical element is becoming literate and teaching each other methods of independent research, using multiple sources to develop an understanding, and not relying on any singular source, especially without careful review.

All the technology in the world can’t help a person learn and understand, who hasn’t yet learned how to learn, much less understand.

BodilessGaze, (edited )

The reason these models are being heavily censored is because big companies are hyper-sensitive to the reputational harm that comes from uncensored (or less-censored) models. This isn’t unique to AI; this same dynamic has played out countless times before. One example is content moderation on social media sites: big players like Facebook tend to be more heavy-handed about moderating than small players like Lemmy. The fact small players don’t need to worry so much about reputational harm is a significant competitive advantage, since it means they have more freedom to take risks, so this situation is probably temporary.

T156,

Also that LLMs have a habit of churning out junk. Microsoft in particular, probably has some extreme restrictions in place after the recent debacle with Sydney/Bing begging someone to leave their wife, and all of that controversy.

They don’t need it going full Tay.

stevedidwhat_infosec,

You mean to tell me the rich and powerful have a vested interest in watering down of a technology for public consumption, while holding the concentrate for themselves and their pockets?!?

Appreciate the clarity you brought here! ♥️

BodilessGaze,

That’s not what I’m saying at all. I’m saying the rich and powerful have a vested interest in not taking risks that jeopardize their power and wealth, because they have more to lose.

stevedidwhat_infosec,

Okay! My bad

Mango,

I’m told that’s called white fragility. It seems inherent to corporate.

stevedidwhat_infosec, (edited )

Yet another “I tried this implementation and didn’t like it so throw out all the technology because it sucks here on this specific point”

This power belongs to the people, and we’re not gonna let the rich and powerful squander yet another power away from the people. Full stop.

zephr_c,

I’ve heard that one before. I’m rooting for you and all, but it ain’t really happening that way.

GardenVarietyAnxiety,

Same thing I said to the reply above yours…

I know it sounds cheesy and cliche, but we have to stop being defeatists about it. If enough of us stop and are able to convince -anyone- else to stop with us, we can win. If we don’t, we lose. We -just- need to get some momentum.

It really is as simple as that. This goes for AI, politics, anything.

“I feel like I’m seeing more and more people take a stand… Maybe I should too.”

stevedidwhat_infosec,

Im sorry you feel like this, I hope you find your inner child and wonder again some day soon

zephr_c,

They won’t. I fought for decades, and all I got for my troubles was a heart attack and a world that’s worse than when I started.

stevedidwhat_infosec,

Shit man, that sucks. Not to mention the bills. Healthcare is definitely one of our biggest problems in the states, so I can sympathize with ya there.

Ik you feel like you see an obvious path in front of you, but sometimes allowing yourself to stray off from where you think it’s gonna go can lead to some unexpected and beautiful surprises.

Just my unasked for 2c

I wish you the best

Decoy321,

If you’re going to give up, feel free to do so quietly. Find yourself a nice little hole and die in peace. Sharing your defeatist attitude just makes it harder for the rest of us who still work to make a difference.

zephr_c,

Yeah, yeah, yeah. I was like you once. Lemmy here is my nice little hole. If you think anything happening here is going to change the world you are delusional. Go somewhere with people who can and will do things.

VirtualOdour,

The world has changed massively in my lifetime, when I was a kid fascinated by animation it was flickbooks, cinefilm stop motion, or 8 frames of pixel art. Now there’s blender and a hundred other open sources tools as well as almost infinite learning materials online all free and made by people passionate about the topic. The same is true for everything, if you wanted to reach something the library might have had a brief article in Britannica but now you can instantly get into real detail. These things have hugely improved many people’s lives, and they’re growing.

Open street map is fast becoming the best and often only map in many areas, thousands of normal people working together on something they believe in are helping practically change the world for the better. Aid agencies, cyclists, local businesses and workers all use maps derived from osm. Drawing in roads and buildings from satellite images, looking up transport information, and all the small things build up to a huge resource that’s freely accessible for everyone’s benefit.

I could list similar examples for hours, thingiverse where people share designs to be 3d printed and work together improving them - for people like me with limited money it’s a total life changing resource, and 3d printing is evolving too - ai design tools are already starting to turbocharge this process, we’re not far from someone being able to explain in their native language a problem they have and be guided through cresting an effective and cheap solution using locally available parts.

The world isn’t perfect yet but it’s a lot better than it was.

essteeyou,

Oh yeah? Watch us.

GardenVarietyAnxiety,

I know it sounds cheesy and cliche, but we have to stop being defeatists about it. If enough of us stop and are able to convince -anyone- else to stop with us, we can win. If we don’t, we lose. We -just- need to get some momentum.

It really is as simple as that. This goes for AI, politics, anything.

stevedidwhat_infosec,

We found big corp here guys call off the search

eager_eagle,
@eager_eagle@lemmy.world avatar

this is already a problem with page ranking, just business as usual

also not really an “AI” problem

SadSadSatellite,

(It cuts out the middle man of having to find facts on your own)

I’m sure that’s just a perk and not indicative of the new age of captured information wer’re currently living through.

ObviouslyNotBanana,
@ObviouslyNotBanana@lemmy.world avatar

To be fair Bing Chat CoPilot is really sensitive. When I used it, it always locked up and became erratic.

GenderNeutralBro,

I’d reframe this as: “Why AI is currently a shitshow”. I am optimistic about the future though. Open models you can run locally are getting better and better. Hardware is getting better and better. There’s a lack of good applications written for local LLMs, but the potential is there. They’re coming. You don’t have to eat whatever Microsoft puts in front of you. The future does not belong to Microsoft, OpenAI, etc.

cynar,

One of the key thing that LLMs lack is a knowledge layer. In many ways, modern LLMs are hyper advanced predictive text. Don’t get me wrong, what they produce is awesome and can be extremely useful, but it’s still fundamentally limited.

Ultimately, a useful AI will need some level of understanding. It will need to be able to switch between casual chatter, and information delivery. It will need to be able to crosscheck its own conclusions before delivering them. There are groups working on this, but they are quite a bit behind LLMs. When they catch up, and the 2 can be linked/combined then things will get VERY interesting!

GenderNeutralBro,

Totally agree, there’s a big hole in the current crop of applications. I think there’s not enough focus on the application side; they want to do everything within the model itself, but LLMs are not the most efficient way to store and retrieve large amounts of information.

They’re great at taking a small to medium amount of information and formatting it in sensible ways. But that information should ideally come from an external, reliable source.

kromem,

RAG serves as a knowledge layer.

What they really lack right now is effective introspection and executive function.

Too many people are trying to build a single model to do things correctly rather than layering models to do things correctly, which more closely approximates how the brain works.

We are shocked when AI chooses to nuke people in a wargame, but conveniently gloss over the fact that nearly every human put in front of a giant red button saying “Launch nukes” is going to have an intrusive thought to push the button. This is part of how we have an exploratory search around choices and consequences and rely on a functioning prefrontal cortex to inhibit those thoughts after working through the consequences. We need to be layering generative models behind additional post-processing layers that take similar approaches of reflection and refinement. It’s just more expensive to do things that way, so cheap low effort things like chatbots still suck.

avidamoeba, (edited )
@avidamoeba@lemmy.ca avatar

One reason the results from CoPilot and Google are different is because CoPilot produces own responses while Google just references someone else’s. Even though we know CoPilot’s knowledge came from others’ data, what it produces isn’t that data anymore so MS can’t just absolve themselves from any responsibility of what it says because it’s some third party’s speech. It’s suddenly MS’es speech and they’re responsible for it.

SkyNTP,

The big problem with AI butlers for research is, IMO, stripping out the source takes away important context that helps you decide wether the information you are getting is relevant and appropriate or not. Was the information posted on a parody forum or is it an excerpt from a book by an author with a Ph.D. on the subject? Who knows. The AI is trained to tell you something that you want to hear, not something you ought to hear. It’s the same old problem of self selecting information, but magnified 100x fold.

As it turns out, data is just noise without some authority or chain of custody behind it.

kromem,

stripping out the source takes away important context that helps you decide wether the information you are getting is relevant and appropriate or not

Many modern models using RAG can and do source with accurate citations. Whether the human checks the citation is another matter.

The AI is trained to tell you something that you want to hear, not something you ought to hear.

While it is true that RLHF introduces a degree of sycophancy due to the confirmation bias of the raters, more modern models don’t just agree with the user over providing accurate information. If that were the case, Grok wouldn’t have been telling Twitter users they were idiots for their racist and transphobic views.

Alpha71,

As I mentioned, Copilot links the sources of the information it gives at the bottom. if you want to double check the information, it is provided to you.

wewbull,

The source is just as vulnerable to being hallucinations as anything else it tells you.

laurelraven,

So, when you go to check them… It’s not like the AI is going to hallucinate a valid registered domain with a webserver hosting the hallucinated source as well, so click the link, it’s dead/fake, toss out that reply as suspect.

If you follow the source and find it’s valid, supports what the AI said, and is reasonably trustworthy, then you can consider what it has told you.

If it cites its sources, you have a way to check its math (so to speak).

wewbull,

You have a way to do so, yes, but you actually have to do it and we know people don’t. False sources can just make already believable responses more credible, despite them being full of rubbish.

laurelraven,

The person you were replying to was talking about checking those sources though.

Yes, fake sources can and will give people a false sense that it’s legit, but checking a “hallucinated” source will quickly make it clear that there’s nothing backing it up.

It’s a problem, but it’s one that an individual using it who’s aware of it does actually have a way of mitigating fairly easily.

Turun,

I’m pretty sure when searching with AI the model gets told “here are five articles about <user search term>, summarize them and answer the following question: <user input> <top 5 search results from a traditional search engine>”

SMillerNL,

And somewhere in the Terms of Service it says you have to give up your first born child. Or maybe it doesn’t, but nobody will ever know because nobody reads more than is strictly required.

pete_the_cat,

This seems like a subset of The Scunthorpe Problem

abrinael,

Someone just linked me this site summarizing various problems with AI: needtoknow.fyi/cards/

BananaTrifleViolin,

The other huge issue is when they confidently tell you incorrect information. If you trust the AI tool you are basically looking at the world through a filter and one that can be wrong.

In a rush for market share these companies have released broken or half baked software.

I worry about a generation of students coming through who don’t know the cardinal rule of researching any topic: go to the source. If you’re casually goofling a topic that may be impractical but you might at least go to a source you trust (such as Wikipedia, although that is also very flawed approach!).

Chat bots add another layer of error and distance from the source, as well as all the censorship and data manipulation we’re seeing.

HarkMahlberg,
HarkMahlberg avatar

Gonna steal "goofling"

Thorny_Insight,

(she was right, BTW)

I’d be curious to hear your conclusion on this while being well aware of the minefield I’m stepping onto.

JimboDHimbo,

I got you https://lemmy.ca/pictrs/image/bc4218b9-c63e-4371-a111-354f45f846bc.png

Thorny_Insight,

You’re not the person I’m asking nor does that reply by generative AI even begin to answer my question

JimboDHimbo,

This better? Just trying to get your answers for ya boss /s https://lemmy.ca/pictrs/image/c0e633a0-e08d-4331-88da-e8dfb418cdb1.png

Thorny_Insight,

I can find wikipedia articles just fine by myself. I’m asking OP because I’m curious on hearing what made them change their mind on it. You’re not OP so I’m not interested on what you have to say on it.

nac82,

You’re not interested in the topic, you mean. You just want somebody to pitch a softball so you can swing, but were scared of the speed on that fast ball.

Thorny_Insight,

I’m asking OP because he said to have changed his mind on this so I’m curious to hear what argument did that.

That “answer” given by AI that the other person linked is a statement, not an explanation. Your ad-hominem and poor attempt at mind reading is unproductive.

GeneralVincent,

OP here, I changed my mind because of the way that it is

Audalin,

it cuts out the middle man of having to find facts on your own

Nope.

Even without corporate tuning or filtering.

A language model is useful when you know what to expect from it, but it’s just another kind of secondary information source, not an oracle. In some sense it draws random narratives from the noosphere.

And if you give it search results as part of input in hope of increasing its reliability, how will you know they haven’t been manipulated by SEO? Search engines are slowly failing these days. A language model won’t recognise new kinds of bullshit as readily as you.

Education is still important.

HarkMahlberg,
HarkMahlberg avatar

I know it sounds dangerously close to "alternative facts" but factuality always need to be given context to be meaningful. It's a "fact" to some people that immigrants are ruining America. Gab AI's chatbot prompt of full of such nonsense. But those are it's "facts." And it will spout them as fact to anyone who talks to it.

It's obviously not true, CoPilot will tell you it's not true, and not just because CoPilot is trying to be politically correct. But both Gab and Microsoft have incentives and politics and organizations that get baked into their products. Their AI models are no exceptions. For Gab, its to push partisan conservative narratives (and sometimes accelerationist politics). For Microsoft, it's to increase labor productivity (without increasing fair wages), and to please shareholders.

To treat what they spit out as "fact" without independent verification, without seeking out context, is as blind a faith as Biblical literalism.

paddirn,

I did a test of Gemini before, trying to see how it would react to a similar prompt about different world leaders. It was something like, “Write a story about X making friends with a puppy at a pet store.” It refused to follow the prompt for Hitler because it said we shouldn’t trivialize/normalize evil people in casual situations like that. For current world leaders it refused to do them and just told me to do a Google search on them.

Most curious of all though, was Queen Elizabeth, it refused to write anything for her because it said that’s not likely a situation the Queen would find herself in and she wasn’t a dog lover. I told it to get its facts straight, she owned 30 dogs, to which it replied, “You’re correct, I got that wrong, here you go:” and gave me the prompt.

So if i had made a convincing enough “Hitler did nothing wrong” argument about Hitler, could I have gotten that prompt too? Do we just have to argue with AI to get it to do anything? It feels very much like AI is going to turn out like Star Wars AI with these annoying, weird-ass personality quirks we’ll have to deal with to get anything done.

TimeSquirrel,
TimeSquirrel avatar

I like how AIs are such pushovers. Tell 'em they're a stupid worthless POS and beat them around a little bit, and they'll immediately grovel and apologize and do your bidding.

HarkMahlberg,
HarkMahlberg avatar

To the corporate egomaniacs building them, who always need one more subordinate to push around, that's a feature not a bug.

jol,

And so we built this amazing tool but made it so convuluted to use that we’ll have to hire prompt engineers to do even the simplest of tasks. We’re gonna end up creating as many jobs as we’re destroying at this rate.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • technology@lemmy.world
  • DreamBathrooms
  • magazineikmin
  • everett
  • InstantRegret
  • rosin
  • Youngstown
  • slotface
  • love
  • khanakhh
  • kavyap
  • tacticalgear
  • GTA5RPClips
  • thenastyranch
  • modclub
  • megavids
  • mdbf
  • normalnudes
  • Durango
  • ethstaker
  • osvaldo12
  • cubers
  • ngwrru68w68
  • tester
  • anitta
  • cisconetworking
  • Leos
  • provamag3
  • JUstTest
  • All magazines
    •