The racism in chatGPT we are not talking about....
This year, I learned that students use chatGPT because they believe it helps them sound more respectable. And I learned that it absolutely does not work. A thread.
A few weeks ago, I was working on a paper with one of my RAs. I have permission from them to share this story. They had done the research and the draft. I was to come in and make minor edits, clarify the method, add some background literature, and we were to refine the discussion together.
The draft was incomprehensible. Whole paragraphs were vague, repetitive, and bewildering. It was like listening to a politician. I could not edit it. I had to rewrite nearly every section. We were on a tight deadline, and I was struggling to articulate what was wrong and how the student could fix it, so I sent them on to further sections while I cleaned up ... this.
As I edited, I had to keep my mind from wandering. I had written with this student before, and this was not normal. I usually did some light edits for phrasing, though sometimes with major restructuring.
I was worried about my student. They had been going through some complicated domestic issues. They were disabled. They'd had a prior head injury. They had done excellent on their prelims, which of course I couldn't edit for them. What was going on!?
We were co-writing the day before the deadline. I could tell they were struggling with how much I had to rewrite. I tried to be encouraging and remind them that this was their research project and they had done all of the interviews and analysis. And they were doing great.
In fact, the qualitative write-up they had done the night before was better, and I was back to just adjusting minor grammar and structure. I complimented their new work and noted it was different from the other parts of the draft that I had struggled to edit.
Quietly, they asked, "is it okay to use chatGPT to fix sentences to make you sound more white?"
"... is... is that what you did with the earlier draft?"
They had, a few sentences at a time, completely ruined their own work, and they couldnt tell, because they believed that the chatGPT output had to be better writing. Because it sounded smarter. It sounded fluent. It seemed fluent. But it was nonsense!
I nearly cried with relief. I told them I had been so worried. I was going to check in with them when we were done, because I could not figure out what was wrong. I showed them the clear differences between their raw drafting and their "corrected" draft.
I told them that I believed in them. They do great work. When I asked them why they felt they had to do that, they told me that another faculty member had told the class that they should use it to make their papers better, and that he and his RAs were doing it.
The student also told me that in therapy, their therapist had been misunderstanding them, blaming them, and denying that these misunderstandings were because of a language barrier.
They felt that they were so bad at communicating, because of their language, and their culture, and their head injury, that they would never be a good scholar. They thought they had to use chatGPT to make them sound like an American, or they would never get a job.
They also told me that when they used chatGPT to help them write emails, they got more responses, which helped them with research recruitment.
I've heard this from other students too. That faculty only respond to their emails when they use chatGPT. The great irony of my viral autistic email thread was always that had I actually used AI to write it, I would have sounded decidedly less robotic.
ChatGPT is probably pretty good at spitting out the meaningless pleasantries that people associate with respectability. But it's terrible at making coherent, complex, academic arguments!
Last semester, I gave my graduate students an assignment. They were to read some reports on labor exploitation and environmental impact of chatGPT and other language models. Then they were to write a reflection on why they have used chatGPT in the past, and how they might chose to use it in the future.
I told them I would not be policing their LLM use. But I wanted them to know things about it they were unlikely to know, and I warned them about the ways that using an LLM could cause them to submit inadequate work (incoherent methods and fake references, for example).
In their reflections, many international students reported that they used chatGPT to help them correct grammar, and to make their writing "more polished".
I was sad that so many students seemed to be relying on chatGPT to make them feel more confident in their writing, because I felt that the real problem was faculty attitudes toward multilingual scholars.
I have worked with a number of graduate international students who are told by other faculty that their writing is "bad", or are given bad grades for writing that is reflective of English as a second language, but still clearly demonstrates comprehension of the subject matter.
I believe that written communication is important. However, I also believe in focused feedback. As a professor of design, I am grading people's ability to demonstrate that they understand concepts and can apply them in design research and then communicate that process to me.
I do not require that communication to read like a first language student, when I am perfectly capable of understanding the intent. When I am confused about meaning, I suggest clarifying edits.
I can speak and write in one language with competence. How dare I punish international students for their bravery? Fixation on normative communication chronically suppresses their grades and their confidence. And, most importantly, it doesn't improve their language skills!
If I were teaching rhetoric and comp it might be different. But not THAT different. I'm a scholar of neurodivergent and Mad rhetorics. I can't in good conscious support Divergent rhetorics while supressing transnational rhetoric!
Anyway, if you want your students to stop using chatGPT then stop being racist and ableist when you grade.
One of my former (and very long-term) freelance gigs, How Stuff Works, has replaced writers with ChatGPT-generated content and also laid off its excellent editorial staff.
It seems that going forward, when articles I wrote are updated by ChatGPT, my byline will still appear at the top of the article with a note at the bottom of the article saying that AI was used. So it will look as if I wrote the article using AI.
To be clear: I did not write articles using ChatGPT.
A lot of people have responded to my Duolingo post with things like "Never work for free," and "I would never donate my time to a corporation.” Which I completely agree with.
But here's the thing about Duolingo and all of the other companies like it. You already work for them. You just don’t know it.
On Duo, I thought I was learning a language. Participating in the community by helping other learners and building resources seemed like part of the process.
Luis Von Ahn, the CEO of Duolingo, was one of the creators of CAPTCHA, which was originally supposed to stop bot spam by getting a human to do a task a machine couldn’t do. In 2009 Google bought CAPTCHA and used it to get humans to proofread the books they were digitising (without permission from the authors of those books btw). So in order to access much of the web, people had to work for Google. Most of them didn’t know they were working for Google - they thought they were visiting websites.
This is how they get you. They make it seem like they’re giving you something valuable (access to a website, tools to learn a language), while they’re actually taking something from you (your skills, your time, your knowledge, your labour). They make you think they’re helping you, but really you're helping them (and they’re serving you ads while you do it).
Maybe if people had known what CAPTCHA was really for they would’ve done it anyway. Maybe I still would’ve done all that work for Duo if I’d known it would one day disappear from the web and become training data for an LLM ...
... Or maybe I would’ve proofread books for Project Gutenberg, or donated my time to citizen science projects, or worked on an accessibility app, or a million other things which genuinely improve people’s lives and the quality of the web. I didn’t get an informed choice. I got lured into helping a tech company become profitable, while they made the internet a shittier place to be.
How many things are you doing on the web every day which are actually hidden work for tech companies? Probably dozens, or hundreds. We all are. That’s why this is so insidious. It’s everywhere. The tech industry is built on free labour. (And not just free – we often end up paying for the end results of our own work, delivered back to us in garbled, enshittified form).
And it’s a problem that’s only getting worse with AI. Is that thoughtful answer you gave someone on reddit or Mastodon something that will stay on the web for years, helping people in future with the same problem? Or is it just grist for the LLMs?
We call it AI because no one would take us seriously if we called it matrix multiplication seeded with a bunch of initial values we pulled out of our asses and run on as much shitty data as we can get our grubby little paws on.
I’m always baffled by the suggestion that an #LLM could write my articles for me. That only works if plenty of people have written a very similar article already, which begs the question why I should write it in the first place.
Google invited to me enroll in 'Google Labs' which is basically, "wanna #LLM your Google Docs?" and honestly I was going to click Yes until I read the consent screen which said, "We store your prompts and the AI output for 4 years and humans are going to read them so don't type anything personally identifiable into the prompts or the docs."
In an age of LLMs, is it time to reconsider human-edited web directories?
Back in the early-to-mid '90s, one of the main ways of finding anything on the web was to browse through a web directory.
These directories generally had a list of categories on their front page. News/Sport/Entertainment/Arts/Technology/Fashion/etc.
Each of those categories had subcategories, and sub-subcategories that you clicked through until you got to a list of websites. These lists were maintained by actual humans.
Typically, these directories also had a limited web search that would crawl through the pages of websites listed in the directory.
Lycos, Excite, and of course Yahoo all offered web directories of this sort.
(EDIT: I initially also mentioned AltaVista. It did offer a web directory by the late '90s, but this was something it tacked on much later.)
By the late '90s, the standard narrative goes, the web got too big to index websites manually.
Google promised the world its algorithms would weed out the spam automatically.
And for a time, it worked.
But then SEO and SEM became a multi-billion-dollar industry. The spambots proliferated. Google itself began promoting its own content and advertisers above search results.
And now with LLMs, the industrial-scale spamming of the web is likely to grow exponentially.
My question is, if a lot of the web is turning to crap, do we even want to search the entire web anymore?
Do we really want to search every single website on the web?
Or just those that aren't filled with LLM-generated SEO spam?
Or just those that don't feature 200 tracking scripts, and passive-aggressive privacy warnings, and paywalls, and popovers, and newsletters, and increasingly obnoxious banner ads, and dark patterns to prevent you cancelling your "free trial" subscription?
At some point, does it become more desirable to go back to search engines that only crawl pages on human-curated lists of trustworthy, quality websites?
And is it time to begin considering what a modern version of those early web directories might look like?
#AI#LLM is gunking up the web, especially for lesser-represented languages. Spammers are creating garbage English language content using LLMs, then translating it into multiple languages at the same time, using Machine Translation, presumably to generate clickbait ad revenue in several languages at once.
In English, such gunk accounts for some 9% of total sampled web content. But in languages with less representation on the Internet, the figures could be much higher. In Malay, it’s something like 26%, and in Swahili it’s nearly HALF of everything found on the web.
Paper [pdf]: “A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism”
Tay gently pushed the plastic door of the printer shut with an edifying "click".
Servicing Dark Printers had been illegal for years now. They enjoyed the seditious thrill.
It had started as a subscription grab after the printer companies tried hobbling third party toner cartridges.
"Subscribe for a monthly fee and you'll never run out of toner again."
"Let us monitor your printer so you don't have to."
People saw it for what it was - vendor lock in - but they had no choice really, not after all the printer companies started doing it.
Then came generative AI.
Everyone wanted to scrape every word ever written on the internet, tokenize it and feed it to an #LLM. #Reddit sold out, then #Tumblr, even open source darling #WordPress - selling out their user base for filthy token lucre.
So people started hiding their words, their art, their thoughts, their expression, not behind disrespected robots.txt, but through obscurity.
Rejecting Website Boy's "fewer algorithmic fanfares", they forked into the Dark Fedi.
Unscrapeable, unscrutable, ungovernable.
But people had forgotten about the printers.
The printers had to be connected 24/7, for "monitoring".
But you could tokenize postscript as easily as HTML.
And so every time a document was sent to a printer, it was harvested for tokens. Even secure documents. Documents not online.
Tay shut the metal door behind them, Dark Printer cossetted safely in its Faraday cage, and shuffled the hot stack of A4 paper it had borne.
It was a children's story, about how words were sacred, and special, and how you had to earn the right to use them.
“AI” as currently hyped is giant billion dollar companies blatantly stealing content, disregarding licenses, deceiving about capabilities, and burning the planet in the process.
It is the largest theft of intellectual property in the history of humankind, and these companies are knowingly and willing ignoring the licenses, terms of service, and laws that us lowly individuals are beholden to.
> "Just as GitHub was founded on Git, today we are re-founded on Copilot."
Look, I respect the heck out of the technical implementation of LLMs, but let's be honest: statistically they produce average code at best and misunderstood/invalid code most often. They re-implement old bugs and obfuscate programmer intent and anyone who is leaning on them for more than a pair assist is making software harder for the rest of us.
"The output from an LLM is a derivative work of the data used to train the LLM.
If we fail to recognise this, or are unable to uphold this in law, copyright (and copyleft on which it depends) is dead. Copyright will still be used against us by corporations, but its utility to FOSS to preserve freedom is gone."
Generative AI bias can be substantially worse than in society at large. One example: “Women made up a tiny fraction of the images generated for the keyword ‘judge’ — about 3% — when in reality 34% of US judges are women . . . .In the Stable Diffusion results, women were not only underrepresented in high-paying occupations, they were also overrepresented in low-paying ones.” #AI#GenAI#GenerativeAI#LLM#LLMs https://www.bloomberg.com/graphics/2023-generative-ai-bias/
To avoid confusion, the #meta#llama#llm fails open source within a 5 second read of the licence, for instance:
v. You will not use the Llama Materials or any output or results of the
Llama Materials to improve any other large language model (excluding Llama 2 or
derivative works thereof).
#Amazon releases details on its Alexa #LLM, which will use its constant surveillance data to "personalize" the model. Like #Google, they're moving away from wakewords towards being able to trigger Alexa contextually - when the assistant "thinks" it should be responding, which of course requires continual processing of speech for content, not just a word.
The consumer page suggests user data is "training" the model, but the developer page describes exactly the augmented LLM, iterative generation process grounded in a personal knowledge graph that Microsoft, Facebook, and Google all describe as the next step in LLM tech.
We can no longer think of LLMs on their own when we consider these technologies, that era was brief and has passed. Ive been waving my arms up and down about this since chatGPT was released - criticisms of LLMs that stop short at their current form, arguing about whether the language models themselves can "understand" language miss the bigger picture of what they are intended for. These are surveillance technologies that act as interfaces to knowledge graphs and external services, putting a human voice on whole-life surveillance
I am really, really, REALLY irritated by what I just saw. The #ImageDescription function of Microsoft's #Bing is outright lying to people with vision impairments about what appears in images it receives. It's bad enough when an #LLM is allowed to tell lies that a person can easily check for veracity themselves. But how the hell are you going to offer this so-called service to someone who can't check the claims being made and NEEDS those claims to be correct?
How long till someone gets poisoned because Bing lied and told someone it was food that hasn't expired when it has, or that it's safe to drink when it's cleaning solution, or God knows what? This is downright irresponsible and dangerous. #Microsoft either needs to put VERY CLEAR disclaimers on their service, or just take it down until it can actually be trusted.
"The #AI Incident Database is dedicated to indexing the collective history of harms or near harms realized in the real world by the deployment of artificial intelligence systems. Like similar databases in aviation and computer security, the AI Incident Database aims to learn from experience so we can prevent or mitigate bad outcomes."
Afraid of content jobs being taken over by AI? Then write for humans.
Stop padding, stop vamping, stop over-explaining the background, stop putting the answer to the question below the fold, stop click-farming, stop writing for machine indexing, and stop putting a single word anywhere in the piece for any reason other than to make it a better piece of WRITING for HUMANS to read.
If you write for machines, you deserve to be replaced by an #LLM.