Wondering if anyone has already started adding malicious LLM prompts to their User Agent strings and hammering sites of companies that might be expected to use "AI" for log analysis. 🤔
@rysiek That requires the software to be written spectularly badly to have any effect? The original story around "email LLM" also sounds very very unrealistic, and the comment chains are weirdly centered around cheerleading privacy-centered email providers. Something doesn't add up.
@sehe no, it only requires the LLM agent to be able to perform any kind of actions at all. And without them, the agent is basically useless.
Thing is, LLMs chatbots have no way of doing "parametrized prompts", so to speak. Prompt injection is very much a thing, but as opposed to good old SQL injection, there's no way to actually properly fix it.
Because, again, no way to do parametrized prompts.
You seem to think writing software "spectacularly badly" doesn't happen often… :blobcatcoffee:
@rysiek Oh I know writing bad software is the norm. However, like sql injection, basic measures are actually not that hard, and indeed as you noted here: the fact that the AI has privileges to perform actions is the real problem here. It also seems that somehow they it is allowed to take prompts from tainted (untrusted) input. I don't see how that would be required for normal AI-agent. Perhaps I need to do some reading, as it could be that LLMs make no distinction between "context" and "prompt"?
Add comment