Little more hands-on with a certain #LLM for some time now.
Once I learned what the "stop sequence" is actually good for, my instinctive ascription of at least a little bit of personality to the thing disappeared immediately.
@apublicimage It appears that one asks the system to produce, say, 100 tokens as a continuation of another 100 tokens of prompt. It then produces 100 tokens - and can stop in the middle of a sentence.
To get a chat one obviously trains with chat data that contains speaker marks like "**Human", "**Bot". It then continues an input like "**Human: Hi!" with 100 tokens, inventing reply, question, reply...
With stop sequence "**Human:" it stops after the first reply, and it feels like a chat.
@apublicimage This is why people call it "autocomplete". I rather call it "autocontinuation". It's a pontentially endless stream of words. That stream is simply cut off after n tokens - or if a certain sequence of characters appears, the stop sequence.
The chat experience comes from a certain application that is built on top of that and plays a little bit of theatre.
Add comment