I doubt it’s only about some Reddit posts. The scrapping was done on the whole web, capturing everything it could. So besides stealing data and presenting it as its own, it seems to have collected some even more problematic data which wasn’t properly protected.
But does leaving your front door open allow one to legally take a picture of the inside from across the street? I'd say scraping is more akin to that than it is theft. Nothing is removed in scraping, just copied
@Hick I have one problem with that in terms of this generative ai. It's similar to when microsoft trained copilot on github data. Of course it was open source code, it was on Microsoft's servers but with this ai revolution you couldn't expect that someone will be able to create such tool. I mean we're randomly leaving our DNA in multiple different places but does it mean we agreed to be cloned once the technology that makes it possible will arrive?
Here is not just scraping though, it is also using that data to create other content and to potentially also re-publish that data (we have no way of knowing whether chatGPT will spit out any of that nor where did it take what is spitting out).
The expectation that social media data will be read by anybody is fair, but the fact is that the data has been written to be read, not to be resold and published elsewhere too.
It is similar for blog articles. My blog is public and anybody can read it, but that data is not there to be repackaged and sold. The fact that something is public does not mean I can do whatever I want with it.
I could read your blog post and write my own blog post, using yours as inspiration. I could quote your post, add a link back to your blog post and even add affiliate links to my blog post.I could be hired to do something like that for the whole day
ChatGPT doesn't get inspired, the process is different and it could very well spit verbatim the content. You can do all the rest (depending on the license) without issues, but once again this is not what chatGPT does, as it doesn't provide attribution.
Just because something is posted online doesn't mean it can be taken a resold. Copyright law prevents that. Of course, copyright law and generative AI is new and gray area.
Likely an unpleasant or possibly infeasible thing to implement, but designing the AI to always be able to “show the receipts” for how it’s formulating any given response could potentially be helpful. Suppose that could result in like a micro-royalties sort of industry to crop up for sourced data being used, akin to movies or TV using music and paying royalties
The way generative AI works is by using things called "tokens". Usually 1 word == 1 token, but compound words would be 2 tokens, punctuation would be a token, things like "-ed" or "-ing" could be tokens, etc.
When you give an AI a prompt, it breaks your response down into tokens. It then finds what tokens were statistically most likely to appear near that content and gives them as a response.
This has been the approach for a while, but the modern breakthroughs have come from layering AIs inside of each other. So in our example, the first AI would give an output. Then a second AI would take that output and apply some different rules to it - this second AI could have a different idea of what a "token" is, for example, or it could apply a different kind of statistical rule. This could be passed to a third AI, etc.
You can "train" these AI by looking at their output and telling it if it was good or bad. The AIs will adjust their internal statistical models accordingly, giving more weight to some things and less weight to others. Over time, they will tend towards giving results that the humans overseeing the AI say are "good". This is very similar to how the human brain learns (and it was inspired by how humans learn).
Eventually, the results of all these AI get combined and given as an output. Depending on what the AIs were trained to do, this could be a sentence/response (ChatGPT) or it could be a collection of color values that humans see as a picture (DALL-E, Midjourney, etc.).
Because there are so many layers of processing, it's hard to say "this word came from this source." Everything the AI did came from a collection of experiences, and generally as long as the training data was sufficiently large you can't really pinpoint "yeah it was inspired by this." It's like how when you think of a dog, you think of all the dogs you've experienced in your lifetime and settle on one idea of "dog" that's a composite of all those dogs.
Interestingly, you can sometimes see some artifacts of this process where the AI learned the "wrong" thing. One example: if you asked an AI what 3 + 4 is, it knows from its experiences that statistically it should say "7". Except people started doing things like asking for what "Geffers + HippoLady" was, and the bot would reply "13", consistently.
It seemed there were these random tokens that the bot kept interpreting as numbers. Usually they were gibberish, but sometimes you could make out individual words being treated as 1 token despite being 2 separate words.
It turned out that if you googled these words, you'd get redirected to a subreddit - specifically /r/counting. The tokens were actually the usernames of people who contributed often to /r/counting. This is one way it was determined that the bot was training on Reddit's data - because these usernames appeared near numbers a lot, the bot assumed they were numbers and treated those tokens accordingly.
Such a detailed response, thank you for that. It walked the line well between keeping it fairly simple but still detailed to understand it.
Because of the complexity and “mystery box” nature of ai for me, it’s hard not to just allow it in my mind to just to consider it as another form of intelligence. But people dismiss it as not intelligent because they have a far better understanding of how the AI has been trained and also in how it came to the results it has. This, VS humans where you’re like “oh I know he went to college in the medical field” but you don’t have as intimate an understanding how how thoughts, ideas and responses are formed because of that obscured source information. Also of course, far more complex with the other impacting factors like chemicals in the body, sleep, mood, experiences and perceptions.
I guess this is a bit of a run on, but it still makes me wonder if it’s just a case of creating enough obscured understanding that allows for consciousness to be as accepted. Not saying that ChatGPT is like a genuine consciousness, but more that it’s the underpinnings or beginnings for something like that. But this is said as someone with absolutely no training in the medical field as well as the artificial intelligence field, so yeah.
it’s hard not to just allow it in my mind to just to consider it as another form of intelligence.
If it makes you feel better, that's probably a biological response that everyone has, to varying degrees. I heard the phrase "textual pareidolia", meaning that if we see text that looks human enough, we'll automatically put a human face on it and want to treat the author like an actual human. Even though it's just a way of creating sentences that mimics human language, and has no form of "intelligence" whatsoever. It has no idea what it's saying and does not understand the meaning of any of the words it's producing. But our lizard brain is still fooled because it sounds good enough. It's like seeing a face in the clouds or Jesus on burnt toast.
Even though I know what it's doing, and can "break" it by making it sound very not-human without too much effort, it's still hard for me not to end a chat session with "Thank you, have a nice day!"
Interesting - kind of weird how in the visual realm there’s the uncanny valley, but I suppose that would be explained by how significant and instinctual vision has played a role in human evolution to detect faces/weird faces etc
When they train a neutral net, all data that it has ever seen is an input to some degree in generating an output, because all inputs contribute to some degree in affecting edge weights, so the answer is "everything I've ever seen".
You are capable of learning higher-level structures and reasoning, and could form distinct memories and associate some memories with those higher-level structures, so in some cases you could remember and name an event that let you build up a piece of reasoning.
So, if you were asked "why did you ground yourself before touching that circuit board", you might say "well, when I was an undergrad, I fried a RAM chip by touching it without grounding myself".
The generative AIs out now are too primitive to and don't reason like that. There's no logic being learned in the way you're thinking of. I guess the closest analog would be if your eyeballs were just wired directly to your cerebellum and had enormous numbers of pictures of flowers flashed at you, each with the word "flower" being said, and then someone said "flower" and recorded the kind of aggregate image that came to mind. All flowers contribute a bit to that aggregate image, but AI-you isn't remembering distinct events and isn't capable of forming logical structures and walking through a thought process.
In fact, if generative AIs could do that, we'd have solved a lot of the things that we want AIs to be able to do and can't today.
And even a human couldn't do that. If I said said "think of a girl" to human-you, maybe you might think of a specific girl or might remember a handful of the individual girls you have seen in life and think that your abstract girl looks more like one than another. But that's still not listing all of the countless girls you have seen that have affected your mental image.
There will probably come a point where we build AIs that can remember specific events (though they won't have unique memory of every event they've seen any more than you do -- a lot of what intelligence does is choose "important" data and throw out the rest, and AIs don't record everything they experience in full any more than you do). And if they could learn to reason, then they might be able to assign specific events. They might misremember, just like a human could, but they could do something of a human's analog of remembering some events, forming logical thought processes, and trying to create some kind of explanation for what events were associated with that thought process. But all that is off in a future where we build something much more analogous to being as capable as a human.
Right, and I suppose if you still tried to charge for use of references to source data, it would then be a weird slippery slope of weighting for which source data the AI was trained on first. How would you say, bill for references to a circuit board if it was trained on things like dictionaries that include “circuit board” as well as of course, more direct references to circuit boards in tech.
Guess it could be some weird percentage, but I don’t think I would welcome that reality
I would rather an AI have that data, instead of any of the Demons of Google, Twitter, Youtube, etc. The AI won't abuse me the way those companies do on a daily basis.
@Craynak_Zero
But that's why they need this ai trained with enormous amount of data. Such ai can be much better in understanding how to keep your engagement how to convince you to buy something etc. As long as it's (not so)open ai connected with Microsoft I'd say it's exactly the same @L4s
Can't convince me of anything. I'm completely immune to such manipulation. Only subhumans are vulnerable, and I don't really care about subhumans or what happens to them, except to laugh at their misery.
What I do care about though, is money being wasted on frivolous lawsuits. The article seems pretty vague on how started this lawsuit too, probably to protect themselves from public scrutiny. The cowards.
If they used private personally identifiable info, then they ought to be able to retrieve that info from gpt. what prompt would i need to get that persons info? If they can't say a prompt to use to get their info, their point is invalid.
The lawsuit alleges OpenAI crawled the web to amass huge amounts of data without people's permission.
So who exactly is keeping people’s data where it can be easily accessed by a trawler without anybody’s permission? Maybe we should be paying attention to that just as well.
Add comment