@cigitalgem@sigmoid.social
@cigitalgem@sigmoid.social avatar

cigitalgem

@cigitalgem@sigmoid.social

software security #swsec machine learning security #mlsec Tech | Life | Music

This profile is from a federated server and may be incomplete. Browse more on the original instance.

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

You can't fix an LLM by red teaming. It does exactly what it was designed to do. Autoassociative predictive word generation.

So what do you prove when you do prompt injection? Not a damn thing.

Always ask this. How does someone FIX what comes out of a pen test? If there is no fix, there is no change in security posture.


https://www.washingtonpost.com/technology/2023/08/08/ai-red-team-defcon/?wpisrc=nl_technology202

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

We just lost another great light of rationalism. Dan Dennett helped get me started in philosophy of mind way back in the late '80s. Dan was right about lots of things. https://dailynous.com/2024/04/19/daniel-dennett-death-1942-2024/

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

systems can leak confidential data in their training set even with a very silly attack. This is a direct and clear issue that applies well beyond the case

https://www.engadget.com/a-silly-attack-made-chatgpt-reveal-real-phone-numbers-and-email-addresses-200546649.html

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

Have a look at the Usenix login; interview featuring myself and the BIML LLM work.

https://berryvilleiml.com/2024/03/15/rik-farrow-interviews-mcgraw-for-login/

cigitalgem, to llm
@cigitalgem@sigmoid.social avatar

Lets do a TOP TEN LLM Risks list

  1. Recursive pollution

Get the full paper here https://berryvilleiml.com/results/

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

It's the data, dummy.

"The AI company, for example, says it has an advantage of having access to X’s trove of posts."

Musk bought twitter for the data pile.

https://www.wsj.com/tech/ai/elon-musks-x-leans-on-his-ai-startup-9038380d

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

My view is that the "API off" and/or "API expensive" decisions at both twitter and reddit have only to do with building a data moat so that LLMs from "outside" not in clear partnership with the pile in question are prohibited from accessing possible training data.

cigitalgem, to random
@cigitalgem@sigmoid.social avatar
cigitalgem, to llm
@cigitalgem@sigmoid.social avatar

Lets do a TOP TEN LLM Risks list

7: Reproducibility economics

Get the full paper here https://berryvilleiml.com/results/

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

It's not just authors anymore. The NY Times sues OpenAI and Microsoft over ML copyright issues.

systems leak training data consistently.

https://www.wsj.com/tech/ai/new-york-times-sues-microsoft-and-openai-alleging-copyright-infringement-fd85e1c4?mod=mhp

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

Today's talk at secappdev was all about the flaw

You do ARA aka threat modelling, right?

Flag in front of the secappdev venue in leuven.

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

Roy is well past his date range

cigitalgem, to llm
@cigitalgem@sigmoid.social avatar

Lets do a TOP TEN LLM Risks list

  1. Data debt

Get the full paper here https://berryvilleiml.com/results/

cigitalgem, to llm
@cigitalgem@sigmoid.social avatar

Lets do a TOP TEN LLM Risks list

6: Poison in the data

Get the full paper here https://berryvilleiml.com/results/

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

Reminder: in my view, recursion pollution is the number one LLM risk by a long shot

https://www.darkreading.com/dr-tech/will-the-ai-arms-race-lead-to-the-pollution-of-the-internet-

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

is an enormous challenge growing faster than we can comprehend https://www.nytimes.com/2023/09/20/technology/chatgpt-dalle3-images-openai.html

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

The "sleeper agents" paper from anthropic is such a complete bullshit I don't even know where to start. Good grief...such terrible "science."

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

NEW Security Ledger podcast features BIML's LLM risk analysis, recursive pollution, and data feudalism. Always a great time chatting with Paul Roberts! @securityledger

https://securityledger.com/2024/02/episode-256-recursive-pollution-data-feudalism-gary-mcgraw-on-llm-insecurity/

cigitalgem, to random
@cigitalgem@sigmoid.social avatar

Dear press people, you can't fix generative AI by blocking prompts. Really. If you need to talk about why that is, call me up. This credulous coverage is just silly.

https://www.cnbc.com/2024/03/08/microsoft-blocking-terms-that-cause-its-ai-to-create-violent-images.html

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

This is absolutely excellent work from a great reporter/fiction author. Spot on.

"A.I.’s errors have an endearingly anthropomorphic name — hallucinations — but this year made clear just how high the stakes can be."

https://www.nytimes.com/2023/12/19/opinion/artificial-intelligence-chatgpt.html

cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

So, what about that NIST AI attack taxonomy? Here's what BIML thinks:

https://berryvilleiml.com/2024/01/23/another-round-of-adversarial-machine-learning-from-nist/

cigitalgem, to random
@cigitalgem@sigmoid.social avatar
cigitalgem, to ML
@cigitalgem@sigmoid.social avatar

I don't believe we can filter our way out of drinking a polluted ocean of training data. https://www.techtarget.com/searchEnterpriseAI/news/366574580/Microsoft-hires-DeepMind-co-founder-amid-Google-Apple-news

cigitalgem, to random
@cigitalgem@sigmoid.social avatar
cigitalgem, (edited ) to random
@cigitalgem@sigmoid.social avatar

New BIML Bibliography entry (under popular press)

https://www.theatlantic.com/ideas/archive/2023/07/godel-escher-bach-geb-ai/674589/

Doug Hofstadter

An excellent view of LLM production as seen by a top cognitive scientist

https://berryvilleiml.com/references/

  • All
  • Subscribed
  • Moderated
  • Favorites
  • megavids
  • InstantRegret
  • magazineikmin
  • modclub
  • Durango
  • Youngstown
  • rosin
  • khanakhh
  • slotface
  • ngwrru68w68
  • mdbf
  • thenastyranch
  • kavyap
  • DreamBathrooms
  • JUstTest
  • tester
  • everett
  • normalnudes
  • GTA5RPClips
  • osvaldo12
  • ethstaker
  • cisconetworking
  • tacticalgear
  • provamag3
  • Leos
  • cubers
  • anitta
  • lostlight
  • All magazines