kellogh

kellogh, 6 hours ago

further, you can also artificially activate these concepts

they have a version of #claude with the “golden gate bridge” concept artificially activated, and so it tries to make everything it says about thr golden gate bridge

https://www.anthropic.com/news/golden-gate-claude

kellogh, 6 hours ago

so now we have a way to interpret and query #LLM responses in a structured format, as well as a control mechanism for driving LLM behavior

this is great news

Bruce Schneier wrote that prompt injection boils down to the fact that data and code pass through the same channel. with this interpretability work, we’re seeing the beginnings of a control channel separated from the data channel — you can control LLM behavior in a way that you can’t override via the data channel

https://www.schneier.com/blog/archives/2024/05/llms-data-control-path-insecurity.html

kellogh, 6 hours ago

this is great work. i’m excited to see where this goes next

i hope #anthropic exposes this via their API. at this point in time, most of the promising interpretability work is only available on open source models that you can run yourself. it would be great to also have them available from #AI vendors

kellogh, 2 hours ago

@ErikJonker yeah, the trustworthiness of LLMs isn’t really an issue. the problem is their profile of trustworthiness didn’t match Google’s existing product

when you create a new product, e.g. perplexity, it’s all new so any feature is positive. when you change an existing product, each feature is a diff from the previous state. so in this case, LLMs made google decidedly worse

if they had launched a totally new product, it probably would have had a mostly positive response

kellogh, 7 hours ago

@Lobrien i can’t get past the toots where AI summary is citing Reddit. its like a scene from Silicon Valley (HBO)