@bornach@masto.ai avatar

bornach

@bornach@masto.ai

I'm an ex-postdoc researcher who was bullied out of academia over a decade ago

I now pursue my interests in
#science #technology #education #art #mathematics

via online content creation that explores ideas in #computer #programming, science #communication, #visualization, #electronics #circuit design, #cardboard #crafts, kinetic sculpture, #synthesizer music, and machine learning

This profile is from a federated server and may be incomplete. Browse more on the original instance.

steely_glint, to random
@steely_glint@chaos.social avatar

Thanks to @saghul for the perfect illustration of the problems with chatGPT:

bornach,
@bornach@masto.ai avatar

@the_moep @steely_glint
It understands kilograms just as well as it understands pounds
https://sharegpt.com/c/vijL1Me
That is, it doesn't understand measurements at all

bornach,
@bornach@masto.ai avatar

@steely_glint @solarisfire @saghul
Bing Chat/Copilot reportedly uses GPT4-Turbo and can search the Internet, yet it doesn't understand that you cannot pour 1 liter into an already full jug
https://masto.ai/@bornach/112201221575789055

bornach,
@bornach@masto.ai avatar

@steely_glint @scribe @saghul
Related to the failures that Yejin Choi found
https://youtu.be/SvBR0OGT5VI?t=4m1s

Tried the jugs example on Copilot the other day. No improvement.
https://masto.ai/@bornach/112201311315573304

bornach,
@bornach@masto.ai avatar

@raganwald @solarisfire
Likely the OpenAI engineers went through the failures that users uploaded to ShareGPT
https://sharegpt.com/c/vijL1Me

And on Reddit
https://www.reddit.com/r/ChatGPT/comments/11rr668/still_doesnt_pass_the_featherlead_test/

Then turned them into microtasks for an annotation company in Nigeria or India to source a better answer from a gig worker
https://m.economictimes.com/tech/technology/indian-gig-workers-toil-at-frontlines-of-ai-revolution/articleshow/109864213.cms

The training data created by the annotation gig industry (AGI) was then incorporated into GPT4 via RLHF

atomicpoet, to random
@atomicpoet@atomicpoet.org avatar

The Thirteenth Floor is set in 2024.

So while I’m watching this VHS tape to get some nostalgia for 1999, this film is speculating about the year I’m living in now.

bornach,
@bornach@masto.ai avatar

@jantzen @atomicpoet
IMHO it is the weakest of the 4 scifi films that came out around that time speculating on simulated worlds:

The Thirteenth Floor
eXistenZ
Dark City
The Matrix

bornach,
@bornach@masto.ai avatar

@atomicpoet @jantzen
Wonder why they cut the ending. Perhaps the twist reveal came too early in the plot so this extended ending just seemed to drag things out.
https://youtu.be/lB17_peD96w

Compare this with how eXistenZ paced their plot twist reveal

18+ urusan, to random
@urusan@fosstodon.org avatar

This is an interesting video:
https://youtu.be/dDUC-LqVrPU

TL;DW We're starting to see early evidence of diminishing returns with our current AI architectures. If this is true, then eventually they start to improve in a logarithmic manner, making superintelligence (at least using our current architectures) impossible to achieve from a practical standpoint. The issue is that we need too much data on specific things for it to perform well on them all.

bornach,
@bornach@masto.ai avatar

@urusan
See also
https://youtu.be/nkdZRBFtqSs
on the implications for post-AI-bubble applications for all the LLM systems that are predicted to fall far short of the hype.

bornach,
@bornach@masto.ai avatar

@dcz @urusan
Probably retracted because peer reviewers later found out that Google had given their AI an unfair advantage through the use of EDA tools (Synopsys suite)
https://www.theregister.com/2023/03/27/google_ai_chip_paper_nature/

Dhmspector, to random
@Dhmspector@mastodon.social avatar

And thus the futile and ultimately fools errand that is “AGI” is exposed…

https://apple.news/Ak8hDr7jRQkCJIAmaUjtH0w

bornach,
@bornach@masto.ai avatar

@Dhmspector
Just predicting the behavior of a single dendrite already requires a deep 5 to 8 layer artificial neural network
https://youtu.be/hmtQPrH-gC4

bornach,
@bornach@masto.ai avatar

@kiki_mwai_mwai @Dhmspector
Apparently the pervading belief among leading AI companies is that there is no need to understand neuroscience. Just need to throw enough training data at a sufficiently deep transformer (or related attention mechanism) and they will get AGI.

There might be a few problems with this approach as highlighted in this [Internet of Bugs] video
https://youtu.be/nkdZRBFtqSs

KathyReid, to stackoverflow
@KathyReid@aus.social avatar

Like many other technologists, I gave my time and expertise for free to #StackOverflow because the content was licensed CC-BY-SA - meaning that it was a public good. It brought me joy to help people figure out why their #ASR code wasn't working, or assist with a #CUDA bug.

Now that a deal has been struck with #OpenAI to scrape all the questions and answers in Stack Overflow, to train #GenerativeAI models, like #LLMs, without attribution to authors (as required under the CC-BY-SA license under which Stack Overflow content is licensed), to be sold back to us (the SA clause requires derivative works to be shared under the same license), I have issued a Data Deletion request to Stack Overflow to disassociate my username from my Stack Overflow username, and am closing my account, just like I did with Reddit, Inc.

https://policies.stackoverflow.co/data-request/

The data I helped create is going to be bundled in an #LLM and sold back to me.

In a single move, Stack Overflow has alienated its community - which is also its main source of competitive advantage, in exchange for token lucre.

Stack Exchange, Stack Overflow's former instantiation, used to fulfill a psychological contract - help others out when you can, for the expectation that others may in turn assist you in the future. Now it's not an exchange, it's #enshittification.

Programmers now join artists and copywriters, whose works have been snaffled up to create #GenAI solutions.

The silver lining I see is that once OpenAI creates LLMs that generate code - like Microsoft has done with Copilot on GitHub - where will they go to get help with the bugs that the generative AI models introduce, particularly, given the recent GitClear report, of the "downward pressure on code quality" caused by these tools?

While this is just one more example of #enshittification, it's also a salient lesson for #DevRel folks - if your community is your source of advantage, don't upset them.

bornach,
@bornach@masto.ai avatar

@mapto @j3j5 @blogdiva @KathyReid

This assumption made by Wolfson:
"they do not reproduce images in their data sets"
is on very shaky ground, especially when it comes to Large Language Models.

Patronus AI found several examples of LLMs generating passages of copyrighted books
https://www.patronus.ai/blog/introducing-copyright-catcher
One might be able to chain together a sequence of text completion prompts to regenerate entire chapters.

bornach,
@bornach@masto.ai avatar

@mapto @j3j5 @blogdiva @KathyReid
Not sure what the relevance of corrupt-and-train is to the legal argument being made here. Wolfson claims "they do not piece together new images from bits of images from their training data" but one could argue that neither is transcoding a Disney movie into a lossy MPEG format. Each frame is regenerated from discrete cosine transforms and motion vectors. Error correction happens during storage. Does that make it fair use?

bornach,
@bornach@masto.ai avatar

@krans @wraptile @KathyReid
"Raise everyone up with the tide" would be releasing their training weights and biases as open source as required by CC-BY-SA but OpenAI has just stated they have no intention of doing this
https://youtu.be/lQNEnVVv4OE

Their lawyers will claim fair use and that their Terms and Conditions mean the user has taken on all risk of any copyright infringement
https://youtu.be/fOTuIhOWFXU

bornach,
@bornach@masto.ai avatar

@highvizghilliesuit @KathyReid
Just do an internet search on Transformers, "Attention is all you need", GPT, BERT, etc. There are many great tutorials covering different levels of detail. This video is more of an overview:
https://youtu.be/Rx-5AGHNu7M

They do in fact encode the copyrighted work into their neural network weights and biases, and can be prompted to regenerate entire passages of text.
https://www.patronus.ai/blog/introducing-copyright-catcher

But it is all linear algebra under the hood

bpaassen, to random
@bpaassen@bildung.social avatar

The last days, I could participate in a Dagstuhl Seminar on Generalization in Humans and Machines. I learned a lot of things, especially one: How weird it is that we expect large language models to generalize to all kinds of tasks. Let me explain. (1/10)

https://www.dagstuhl.de/seminars/seminar-calendar/seminar-details/24192

bornach,
@bornach@masto.ai avatar

@bpaassen
We laymen think it should work because of scifi movies where
Johnny 5 reads all the encyclopedias and becomes sentient
https://youtu.be/WnTKllDbu5o

bornach,
@bornach@masto.ai avatar

@bpaassen
I was skeptical when the AI companies refused to reveal what was in the training data and seemed uninterested in determining whether their LLM was figuring things out for itself or was simply regurgitating an answer that got scraped into the dataset.

So taking a lead from Yejin Choi
https://www.ted.com/talks/yejin_choi_why_ai_is_incredibly_smart_and_shockingly_stupid?language=en

I tried prompting with well known FAQ puzzles but with slight changes that invalidated the stock answer. Didn't take long to confuse the LLM
https://masto.ai/@bornach/112207324622232774

bornach, to OpenAI
@bornach@masto.ai avatar

All your GPUs are belong to #OpenAI

https://youtu.be/lQNEnVVv4OE
Matthew Berman describes how #SamAltman is gunning for #RegulatoryCapture of the
#AI market

#GPU #monopoly #ArtificialIntelligence #generativeAI #ClosedSource

yassie_j, to random
@yassie_j@labyrinth.zone avatar

Here are some common acronyms used by tech companies in 2024, and what they mean.

AI = Always Inaccurate

AGI = A Guy in India

LLM = Large Lying Model

GPT = Great at Producing Trash

bornach,
@bornach@masto.ai avatar

@yassie_j
AGI = Annotation Gig Industry

https://m.economictimes.com/tech/technology/indian-gig-workers-toil-at-frontlines-of-ai-revolution/articleshow/109864213.cms

ben, to random
@ben@m.benui.ca avatar

Stack Overflow announced that they are partnering with OpenAI, so I tried to delete my highest-rated answers.

Stack Overflow does not let you delete questions that have accepted answers and many upvotes because it would remove knowledge from the community.

So instead I changed my highest-rated answers to a protest message.

Within an hour mods had changed the questions back and suspended my account for 7 days.

Diff view of a stack overflow question showing it being changed from the original text to a protest message, then being changed back again by a mod. Protest text reads: Why does OpenAI get to profit from our work? I have removed this question in protest of Stack Overflow's decision to partner with OpenAI. This move steals the labour of everyone who contributed to Stack Overflow with no way to opt-out. OpenAI has a history of flooding the web with inaccurate information and have explicitly stated that they will never pay creators for their work.

bornach,
@bornach@masto.ai avatar

@hunterhacker
@wuppy @ben
Yup. OpenAI refuses to reveal what was in their training data.
You may be thankful for an answer to a question but thankful to whom? ChatGPT generates answers claiming it as its own creation and OpenAI gets the credit. At least pre-2023 search engines directed you to the original source.

When asked to create a new game that never existed before, ChatGPT regurgited someone else's game idea and gave it a different name.

https://gizmodo.com/chatgpt-copy-sumplete-puzzle-game-summer-rullo-1850212198

bornach,
@bornach@masto.ai avatar

@andrewfelix @mighty_orbot @ben
And their software is laundering the original source of the information from which their AI training data was derived. Doesn't the original author deserve some credit for when ChatGPT regurgitates a lossy paraphrasing of a post scraped from the Internet?

jasonkoebler, to random
@jasonkoebler@mastodon.social avatar

The publisher of a small imprint of roleplaying games magazines/speculative fiction shuts down after 22 years because their submissions have been flooded with AI to the extent they cannot wade through them:

“The problem with AI is the people who use AI. These are people who think their ‘ideas’ are more important than the actual craft of writing, so they churn out all these ‘ideas’ and enter their idea prompts and think the output is a story.”

https://www.404media.co/bards-and-sages-closing-ai-generated-writing/

bornach,
@bornach@masto.ai avatar

@13hannes11 @jasonkoebler
Via Yannic Kilcher https://youtu.be/GtveKYXYo_0?t=13m47s

bornach, to ai
@bornach@masto.ai avatar

[Cold Fusion] on the real threat that AI poses to human music makers
https://youtu.be/wgvHnp9sbGM

#AI #music #Suno #Udio #NickBeato

bornach,
@bornach@masto.ai avatar

And released within hours of each other comes Adam Neely on how hard it will be for to pass a Direction Test
https://youtu.be/N8NyEjB_XeA
It is almost like these two coordinated their releases yet they never acknowledge each other's existence

baldur, to random
@baldur@toot.cafe avatar

“Why Would I Buy This Useless, Evil Thing? - Aftermath”

This x1000. Just… why? https://aftermath.site/why-would-i-buy-this-useless-evil-thing

bornach,
@bornach@masto.ai avatar

@dr2chase @futurebird @baldur
Some of the chips will be analog
https://www.eetimes.eu/could-ibms-ai-chip-reinvent-deep-learning-inference/
But so far the analog and low precision tensor processors will only bee doing inference. The training of deep learning models still has to be on high precision GPUs.

bornach,
@bornach@masto.ai avatar

@futurebird @baldur
In spite of the Teenage Engineering magic dust, the usability sucks
https://youtu.be/ddTV12hErTc

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • Leos
  • tsrsr
  • DreamBathrooms
  • thenastyranch
  • magazineikmin
  • hgfsjryuu7
  • Youngstown
  • InstantRegret
  • slotface
  • khanakhh
  • rosin
  • ngwrru68w68
  • kavyap
  • PowerRangers
  • normalnudes
  • tacticalgear
  • cisconetworking
  • everett
  • vwfavf
  • GTA5RPClips
  • osvaldo12
  • Durango
  • mdbf
  • modclub
  • tester
  • cubers
  • ethstaker
  • anitta
  • All magazines
    •