We “know everything about it” because it’s not that complicated.
You don’t need to process every individual step a search algorithm has to understand how it works. LLMs are the same thing. They’re just a big box of weighted probabilities. Complexity is more than just having a really big model.
We have bits and pieces of a lot of parts, but are nowhere near a complete understanding of any of them. We kind of know how neurotransmitters work, we kind of know how hormones work and interact with those neurotransmitters, we mostly know how individual neurons fire, we kind of know what different parts of the brain do, we kind of know how the brain adapts to physical damage…
We don’t know any of the algorithms it follows. What we do know that it’s a hell of a lot of interconnected parts, and they’re all following very different rules.