kennwhite,

Incredible research at BlackHat Asia today by Tong Liu and team from the Institute of Information Engineering, Chinese Academy of Sciences (在iie.ac.cn 的电子邮件经过验证)

A dozen+ RCEs on popular LLM framework libraries like LangChain and LlamaIndex - used in lots of chat-assisted apps including GitHub. These guys got a reverse shell in two prompts, and even managed to exploit SetUID for full root on the underlying VM!

image/jpeg
image/jpeg

kennwhite,

TL;DR: The most popular chat-assisted app frameworks aren't even doing basic process sandboxing or sane file/network isolation. We are still very much in the early infancy of security maturity with current gen LLMs.

kennwhite, (edited )

Liu et al's preprint: https://arxiv.org/pdf/2309.02926.pdf
BlackHat abstract: https://www.blackhat.com/asia-24/briefings/schedule/index.html#llmshell-discovering-and-exploiting-rce-vulnerabilities-in-real-world-llm-integrated-frameworks-and-apps-37215

kennwhite,

Tong's Google Scholar for related work: https://scholar.google.com/citations?hl=en&user=egWPi_IAAAAJ

teajaygrey,
@teajaygrey@rap.social avatar

@kennwhite This is a good thing.

Perhaps some folks can rm -rf / with abandon and nip this BS in the bud.

sigh Alas, I doubt anyone is that forward thinking anymore.

wonka,
@wonka@chaos.social avatar

@teajaygrey You'd need to delete the system that provisions the system that provisions the VMs...
@kennwhite

mjgardner, (edited )
@mjgardner@social.sdf.org avatar

@kennwhite Looks like we’re at the “Matt’s Script Archive” level with frameworks.

The difference is that Matt Wright was a high school student in 1995 when he launched MSA and its infamously exploitable FormMail script.

yaleman,
@yaleman@mastodon.social avatar

@kennwhite sounds like devs need to read the OWASP Top 10 for LLM apps 😄 https://owasp.org/www-project-top-10-for-large-language-model-applications/

byteborg,
@byteborg@chaos.social avatar

@kennwhite
What could possibly go wrong? 🤷 /s

loke,
@loke@functional.cafe avatar

@kennwhite I was in the room at that session. It was quite interesting. It was just a few hours ago, are you there too?

kennwhite,

@loke yes. I'm on the BH Asia review board and here for the rest of the conference.

loke,
@loke@functional.cafe avatar

@kennwhite you're not on stage right now? 😃

phryk,
@phryk@mastodon.social avatar

@kennwhite Oh my, that's glorious! 😂

Infoseepage,
@Infoseepage@mastodon.social avatar

@kennwhite There truly is a XKCD for everything.

https://xkcd.com/327/

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • DreamBathrooms
  • magazineikmin
  • InstantRegret
  • hgfsjryuu7
  • vwfavf
  • Youngstown
  • slotface
  • thenastyranch
  • ngwrru68w68
  • rosin
  • kavyap
  • PowerRangers
  • Durango
  • khanakhh
  • anitta
  • mdbf
  • tacticalgear
  • ethstaker
  • modclub
  • osvaldo12
  • everett
  • tester
  • cubers
  • GTA5RPClips
  • normalnudes
  • Leos
  • cisconetworking
  • provamag3
  • All magazines
    •