In our book Calling Bullshit, Jevin West and I provide the following definition.
#Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade by impressing and overwhelming a reader or listener, with a blatant disregard for truth and logical coherence.
Notice that this is what #LargeLanguageModels are designed to do. They are designed with a blatant disregard for logical coherence in that they lack any underlying logic model whatsoever.