I'm really concerned about the effect "generative AI" is going to have on the attempt to build a copyleft/commons.
As artists/coders, we saw that copyright constrains us. So we decided to make a fenced-off area where we could make copyright work for us in a limited way, with permissions for derivative works within the commons according to clear rules set out in licenses.
Now OpenAI has made a world where rules and licenses don't apply to any company with a valuation over $N billion dollars.
@mcc I'm waiting for the programmer equivalent of nightshade.
"This is a perfectly functional library for X, I've just made it delete your entire C drive, but DON'T WORRY you can remove the offending code in file ev6.cpp, line 45"
@glassbottommeg@mcc That's the thing, though. The text models don't retain the expression of any text, just statistics on the probability of tokens following tokens.
So, yeah, you can do it - but a lot of people (dozens? hundreds? thousands?) have to use the same trick the same way, otherwise the model could never reproduce it.
@Dancer@mcc true, I guess there's kinda no way of effectively poisoning code. Since you can't introduce single pixel changes that do nothing to the image visually but corrupt the AI.
In a world where copyleft licenses turn out to restrict only the small actors they were meant to empower, and don't apply to big bad-actor "AI" companies, what is the incentive to put your work out under a license that will only serve to make it a target for "AI" scraping?
With NFTs, we saw people taking their work private because putting something behind a clickwall/paywall was the only way to not be stolen for NFTs. I assume the same process will accelerate in an "AI" world.
@pinkdrunkenelephants The licenses already bar this because they govern derivative works. If you can make the derivative work non-derivative by defining it "AI", then if we add a nonsense clause banning "AI", the AI companies can simply rename "AI" to "floopleflorp" and say "Ah, but your license only bans 'AI'— it doesn't ban 'floopleflorp'!"
@pinkdrunkenelephants But AI is not AI. The things that they're calling "AI" are just some machine learning statistical models. Ten years ago this wouldn't have been considered "AI".
@pinkdrunkenelephants The definition behind the law is, again, decided by humans, who are capable of inconsistency or poor decisions. Rape is legal in New York because rape there is legally defined by the specific use of certain specific genitals. See E. Jean Carroll v. Donald J. Trump
@mcc And no one accepts that because of what I'm saying. A rose by any other name would smell as sweet. People need to start recognizing that fact. That's the only way things will change.
@pinkdrunkenelephants Well, per my belief as to the meaning of words, ML statistical models are derivative works like any other, and my licenses which place restrictions on derivative works already apply to the ML statistical models
@pinkdrunkenelephants@mcc That doesn't work if copyright itself doesn't apply to AI training, which is what all those court cases are about. Licenses start from the assumption that the copyright holder reserves all rights, and then the license explicitly waives some of those rights under a set of given conditions.
But with AI, it's up in the air whether a copyright holder has any rights at all.
@datarama@mcc I don't see how it would be up in the air. Humans feed that data into AI and use the churned remains so it's still a human violating the copyright.
@pinkdrunkenelephants@datarama Because humans also are the ones who interpret and enforce laws and if the government does not enforce copyright against companies which market their products as "AI", then copyright does not apply to those companies.
@pinkdrunkenelephants@mcc In the EU, there actually is some legislation. Copyright explicitly doesn't protect works from being used in machine learning for academic research, but ML training for commercial products must respect a "machine-readable opt-out".
But that's easy enough to get around. That's why eg. Stability funded an "independent research lab" who did the actual data gathering for them.
@datarama I consider this illegitimate and fundamentally unfair because I have already released large amounts of work under creative commons/open source licenses. I can't retroactively add terms to some of them because the plain language somehow no longer applies. If I add such opt-outs now, it would be like I'm admitting the licenses previously didn't apply to statistics-based derivative works
@mcc@pinkdrunkenelephants@datarama In reality no, it is not in the air. OpenAI and the rest are in violation of copyright. If it were you and me, the media companies would be getting ready to make an example of us.
It is only "up in the air" because the laws don't apply to wealthy powerful people
I keep seeing the argument that AI doesn't do what we think it does and that it is truly generating completely novel content and not copying pixel for pixel.
And, because humans and machines don't work in the same way, this argument is used to say that copyright isn't violated.
But, that's nonsense. Creators didn't want nonhumans or corporations taking their work without asking or compensating.
@hugoestr@mcc@pinkdrunkenelephants@datarama It's just that the companies dance back and forth between the mask provided by the tech in order to say, "you can't legally prove any of this because the inner workings of the machine are mysterious and function beyond the human realm".
And, to some extent it breaks the legal system because people are like, "maybe???"
There's definitely a layer beyond the financial at play. There's a nervousness in the presence of "mysterious" technology.
@clarablackink@hugoestr@pinkdrunkenelephants@datarama it makes me think of the era before the 2008 housing crash, when investment firms were hiring physicists as "quants" to make formulas for bundling mortgages into CDOs. The pricing formulas were absurdly complicated and were supposed to make investments that couldn't fail. And the formulas never worked, and weren't meant to. Their actual and only purpose was only to be so complicated people would stop thinking and treat them as magic
@mcc it's kinda gross that the only (current) way to meaningfully and tangibly refuse to be exploited by the mass commercialised theft of the commons is to, well, commercialise the commons.
@mcc although if there's an angstrom-thick silver lining to this whole thing, it's that it has proved incontrovertibly that copyright law was only ever intended to be used as a cudgel by the wealthy and powerful, and never to protect the rights of the individual artist.
@mcc im wondering if the broader art and design worlds will end up in a similar situation to where industries like fashion and jewelry already are, where plagarism is essentially an expectation for the designers working there
@mcc "the legal system is ultimately a weapon wielded by those with more capital against those with less" is of course the punchline after every movement that has tried to use legal mechanisms like licenses to enact social change. it'd be nice if there were some deep pan-institutional awareness of and correction for this.
@mcc Did copyleft licenses ever meaningfully restrict the behavior of large corporations? Licenses are effectively a statement of intent with respect to future litigation, and if the copyright holder is not willing or able to actually perform that litigation, everyone gradually understands that this is a Mexican standoff where one side's guns aren't loaded.
@mcc IIRC, Sony did it much earlier. I cannot even find any record of this, but as I recall, Sony distributed a modified version of GCC as part of their early Playstation SDKs, in a way which clearly violated the GPL. FSF found out somehow, and the result was just that Sony said "oops, our bad, we forgot to contractually forbid members of our SDK program from talking to you" and then later switched to LLVM.
@mcc gosh I hope I'm not spreading misinformation here, my memory of this was that it was a pretty pedestrian corporate interaction not some weird urban legend, but it's bothering me that I can't find any news coverage
@glyph@mcc So the PlayStation thing was basically the way Cygus did compiler port sales. The Long and short of it is they'd do a gcc target for a given CPU and agree to not release it, then distribute it only to the customer, but still under the GPL, with the caveat that the contracts said they retained all the rights and could use the source code in other ports, and I think it may have also had a timeout. Then, \
@glyph@mcc Sony distributed it to game devlopers, who were under their own contracts and NDAs with Sony and received this compiler as a binary and with no right to distribute it. The contractors didn't receive it under the GPL, but Cygnus still had their own copy that would eventually be upstreamed.
This sort of thing has always been part of the business of making and selling GPLed software.
And also the dig at Red Hat above is still just plain not what actually happened.
@vathpela@mcc “didn’t receive it under the GPL” is doing a lot of work there. Like presumably portions of it were copyrighted by the FSF? Were they a party to this contract somehow? Were they okay with binary proprietary licensing?
@glyph@mcc so there are two parts here - 1) Cygnus owned the code they wrote (which wound up being a strikingly large amount of the compiler) until they contributed it back to the FSF, (though they did have a blanket agreement which did that), and ... \
@glyph@mcc 2) in terms of software licensing, as I understand it (and let's be sure: IANAL) if you work for a company and they for instance get a copy of Visual Studio and give it to you, Microsoft has given it to them under whatever license, but your employer has more or less just loaned it to you. You don't have that software licensed to you, your employer does. And that's something that can be established through non-employment contracts as well.
@vathpela@mcc also not a lawyer, but this sounds novel and bizarre to me. "licensed to" is a shorthand; the license is a limited right to perform copies that would otherwise be considered infringement, which presumably an employee acting on behalf of a company would need in exactly the same way as the company itself would; everybody involved needs the right to make these copies
@vathpela@mcc (I mean, just kidding, as long as the MAI v. Peak / Cartoon Network v. CSC circuit split holds this is all gibberish in US law and I have zero idea how it would hold up in Japan)
@vathpela@glyph This (conveying code from company A to B to C while claiming it was not "distributed") seems to violate the plain language of "copying" or "distribution" as it would have been understood by people who place their code under GPL2, and I doubt a court would sign on with it if this were tested. In GPL3 "distribution" is replaced with a more fine-grained definition of terms concerning copying, and so if this loophole was ever real, it's probably shut now.
@vathpela@glyph At any rate if the behavior in question has been halted for unrelated reasons it seems hard to come to a firm conclusion about whether it was entirely legal?
@mcc@glyph I think that's a fair conclusion. Also worth noting that there are other slightly different ways they could have done it that are even more confusing (and maybe they did, I wasn't ever in that part of the business). Like, what if they only ever gave the customers patches, and then rented them out a consultant who downloaded, patched, and compiled gcc for them? It's exactly the same in the end, but a largely different thing being handed over.
@vathpela@mcc thanks again for stepping in so I wasn't spreading around false claims. I still think the whole scheme sounds confusing and dubious, but now at least I know where to go to do research to understand it better in the future!
@vathpela@glyph Hrm, well, now that you mention it, I think there might actually be a very dangerous patch distribution loophole to the GPL, I'm willing to hear that argument. Especially since I've ALSO seen an almost identical loophole used by people who ADVOCATE the gpl (see "LAME"..)
@mcc@glyph Well, I don't think lawyers or the courts see it that way, but again I am not a lawyer so I won't belabor that point any more. Maybe as @luis_in_brief, I hear he is a lawyer, just not your lawyer or my lawyer ;)
@vathpela@mcc@luis_in_brief ultimately the nature of the debate here just proves the point I was making :). We are arguing over what precise powers that a particular genie would have if it were ever released, but the reality is that the genie gets its power from the tens of millions of dollars that would need to be spent to release it in the first place, and we will never find out so it doesn't matter in any practical sense
@vathpela@mcc@glyph ugh, I’m a lawyer but despite recent appearances I have a day job, so I should really get back to my work of (checks todo list) writing up more opinionating on xz, oh gooooodddddddddd
@luis_in_brief@vathpela@mcc we'd all better stop posting so we can get back to our respective jobs, which are checks notes also posting, mostly about the same topics, just with a slightly different emphasis
@vathpela@glyph I find in the text of GPL2: "You may not impose any further restrictions on the recipients' exercise of the rights granted herein."
If Sony [or whoever else] made someone sign an NDA saying they could not distribute the GPLed software they gave them, Sony was violating the GPL, and therefore did not have the right to distribute the modified GPLed code in question.
@glyph@mcc folks with secret platforms have been able to distribute GPL code under assorted NDAs, as an external governing contract covers that - and the GPL covers that case.
@mcc@glyph I think that’s a little unfair to red hat (it’s just more inconveniently available now) but as a general matter, yeah. In particular:
1️⃣ copyleft has always been mostly in practice less legal contract, more social contract (though there are important exceptions); and
2️⃣ because it is ~ impossible to revise, over time copylefts almost always become less contractual because lawyers for users are constantly shrinking them, while lawyers for creators are rarely expanding them.
@mcc@glyph like, GPLv2 to lawyers in the late 1990s: THIS IS A BOOGEYMAN THAT WILL EAT YOUR CHILDREN.
GPLv2 to lawyers in the present: oh, yeah, I learned that in second-year copyright, including the standard weaknesses/limitations, and I know who to speed-dial if I need to get details on how to exploit it.
Exact same text, very different real world impact.
@luis_in_brief@mcc I think the best understanding of the Red Hat issue is r0ml's "freedom from source code":
""" the largest Free Software business, Red Hat, has always had “freedom from source code” as its business model. A business pays Red Hat with the same licensing scheme as they would for any proprietary commercial operating system, in exchange for which Red Hat frees them from the inconvenience of needing to be exposed to the source code """
@glyph
Yes: it's why Linux got massively popular quickly, with open source contributions from notable large-corps, while BSD derivatives just spawned proprietary forks @mcc
@chrisjrn@mcc Is there a good history of this somewhere? My impression of this process is that it was way more complicated than that, particularly that it had way more to do with process than licensing. But I would certainly love to be wrong about this
@glyph@chrisjrn I do think there were some other complications pushing people away from serious use of BSD during this time. But I think Christopher's argument could be made convincingly.
@mcc
The fact that those complications were overcome (eventually) suggests that with the right early adoption and contribution patterns they could have been overcome (expediently), but something got in the way of people wanting to contribute 🤔 @glyph
@chrisjrn@glyph Well, the "complications" I'm referring to were the lawsuit, and the extremely awkward "advertising" clause in the BSD license that was not removed until 1999. Those things were just going to happen over a certain time period either way (you could maybe even argue without Linux blowing up the advertising clause might have stayed in BSD longer…)
@mcc
Fair assessment. Certainly those affected community BSD forks, but I find myself wondering how that threat would have impacted IBM (who were distributing BSD-compatible AIX and still contributing to Linux at the same time). @glyph
@glyph
I'd need to go looking for sources, but my recollection was that Linus was convinced to change licenses from something hand-spun to GPLv2 very early in the process, because legally unsound licensing was a barrier to contribution.
Process was a differentiator, but FreeBSD took a similar process approach and got a headstart from a more technically mature initial codebase. @mcc
Some jerks did mass scraping of open source projects, putting them in a collection called "the stack" which they specifically recommend other people use as machine learning sources. If you look at their "Github opt-out repository" you'll find just page after page of people asking to have their stuff removed:
…but wait! If you look at what they actually did (correct me if I'm wrong), they aren't actually doing any machine learning in the "stack" repo itself. The "stack" just collects zillions of repos in one place. Mirroring my content as part of a corpus of open source software, torrenting it, putting it on microfilm in a seedbank is the kind of thing I want to encourage. The problem becomes that they then suggest people create derivative works of those repos in contravention of the license. (2/2)
So… what is happening here? All these people are opting out of having their content recorded as part of a corpus of open source code. And I'll probably do the same, because "The Stack" is falsely implying people have permission to use it for ML training. But this means "The Stack" has put a knife in the heart of publicly archiving open source code at all. Future attempts to preserve OSS code will, if they base themselves on "the stack", not have any of those opted-out repositories to draw from.
@mcc i feel like we need llm opt out considerations in foss licenses tbh, then host code off github and nothing changes? Hard to enforce idk unlikely politicians will get it right, maybe the ftc will get lucky?
@josh@mcc Either copyright doesn't apply and then whatever you put in your license doesn't matter, or copyright does apply and then the existing copyleft licenses are enough.
@clacke@mcc thats a strong argument, but I feel like companies are using a loophole in interpretation of copyleft for AI, aren’t some of the companies excluding licensed art from datasets now?
@josh@clacke One of the fundamental problems with large-model AI is that it is so difficult to determine how they were created, what it is they're doing once you send off a request to the API, whether it's a computer program at all or a poorly-paid person in the global south typing very quickly.
@josh I don't like this because (1) it means GPL2 is dead, and (2) it feels like admitting that an AI opt-out is something we specifically needed. Meanwhile, machine transformation of my work is something I generally want, I just want the license to be observed.
Like, heck, how am I supposed to rely on my code getting preserved after I lose interest, I die, BitBucket deletes every bit of Mercurial-hosted content it ever hosted, etc? Am I supposed to rely on Microsoft to responsibly preserve my work? Holy crud no.
We want people to want their code widely mirrored and distributed. That was the reason for the licenses. That was the social contract. But if machine learning means the social contract is dead, why would people want their code mirrored?
@emaytch@mcc So there's a lot of stuff that Paul Graham says that I don't agree with (these days; used to be pretty bought in), but I think the point he made about the nature of copyright and patent protection ages ago rings true.
Paraphrasing without citation because I'm not going to go crawling around to find it right now: the alternative to IP protection isn't a magical utopia of shared ideas... It's guilds and secret knowledge protected with violence. We already tried society without intellectual property protection.
@mcc I have generally come to the conclusion that this is an intended effect. All the things you feel compelled to do for the good of others, in an ordinarily altruistic sense, are essentially made impossible unless you accept that your works and your expressions will be repackaged, sold, and absorbed into commercialised datasets.
The SoaD line "manufacturing consent is the name of the game" has been in my head a lot lately.
@gsuberland@mcc One almost wonders if the end-game is to stop pulling and try pushing.
Maybe instead of trying to claw back data we've made publicly crawlable because "I wanted it visible, but not like that" we ask why any of these companies get to keep their data proprietary when it's built on ours?
Would people be more okay with all of this if the rule were "You can build a trained model off of publicly-available data, but that model must itself be publicly-available?"
@mark@gsuberland In my opinion, a trapdoor like "okay, well if copyright doesn't apply to the training data you stole, your model isn't copyrightable either" is no good. The US Gov has already said GenAI images and text are not copyrightable. It doesn't help. The thing about generative AI is it inherently takes heavy computational resources (disk space, CPU time, often-unacknowledged low-wage tagging work). Therefore, as a tool, it is inherently biased toward capital and away from individuals.
@mark@gsuberland If we say "AI is a new class of thing that is outside the copyright regime entirely", that is not a level playing field. The tool is designed in a way it inherently serves the powerful. "Machine learning models are inherently open" is the exact model I am afraid of— a world where copyright is something that applies to actors who have less than some specific amount of money, and anyone with more than that specific amount of money is liberated from it.
@mcc@mark yes. the only real push back solution that levels the playing field would be to say that you are not allowed to unilaterally make money off it, which essentially just falls back to enforcing copyright law against the rich, which... yeah, exactly the problem.
Even if, say, GPT-4 wasn't covered by copyright, so what? Even if you could get it out of OpenAI's data centres in the first place, you couldn't run it with reasonable performance. And you certainly couldn't retrain it.
@datarama@mcc@mark@gsuberland there is one upside to forcing these models to be open and it's that it removes one of the, of not the primary, incentives in developing them in the first place. Yes, they could still sell its execution as a service, but if they lose control of the model itself, it becomes a considerably less profitable endeavor.
Let's say that tomorrow, a judge rules that GPT-4 is not covered by copyright. What has actually changed? OpenAI isn't compelled to share it with anyone, and it's too big for anyone except large and wealthy corporations to actually do anything with.
Sure, you couldn't get sued if you got a bittorrent of it somehow. But you're not getting a bittorrent of a 1.76 trillion parameter neural network anyway.
@datarama@oblomov@mcc@mark and you sure as shit can't afford a whole rack of H200 cards to make use of it, even if you and all your friends pitch in. it's only useful with people who have the capital to wield it.
@SoniEx2 If people feel they are being exploited, they're going to be less likely to do free labor. Previously, copyleft licenses were one way of assuring people they would not be exploited. Now corporations have us on notice they will no longer be honoring copyleft licenses. So people will possibly do less free labor"
"But what about the people being hurt by withholding your labor?" Will not be convincing. You're suggesting people are causing "hurt" by doing nothing.
@SoniEx2 In the thread above, I am not discussing the way I personally plan to behave in future but rather describing a thing I believe people in general will likely act. Trying to convince me, personally, of anything will not prevent this overall trend. You will have to respond to each potential copyleft contributor personally.
@mcc There is no such incentive. There is a very, very strong incentive (namely, not wanting to empower the worst scumbags in tech) to not share your work publicly anymore.
This, to me, is the most harmful effect so far of generative AI.
@mcc I like that it 'breaks' copyleft because depending on cops to create a social good was always a losing proposition. The only question was to when, and now is as good a time as any.
@mcc And then the companies and people who cynically exploit our belief in the future to promote these scams have the audacity to tell us that we’re pessimistic about technology. We’re not! We love tech! We hate the bullshit, the scams.
@mcc at least we still have Minecraft and no man sky as popular examples of the last one, so with any luck, once the ML scam dies, people will forget about it and only remember it being used for good
@mcc The sickening thing is, each of these does have potential good things to them. But naturally greedy people twisted them to the most evil possible course.
Cryptocurrency could have created an independent system worldwide with less volatility.
Metaverse could have meant transcending current limitations. But FB exists.
LLMs could have allowed neurodivergent people such as myself who cannot create to finally be able to do so. Now we can't do anything with it without being hated.
Add comment