@kellogh I use that exact analogy. And emphasize that we certainly do use and need System 2 at least occasionally. At some point, human-like reasoning must use symbols with definite, not probabilistic, outcomes. Can that arise implicitly within attention heads? Similar to embeddings being kinda-sorta knowledge representation? I mean, maybe? But it still seems hugely wasteful to me. I still tend towards neuro-symbolic being the way.
kellogh