knowprose, 13 days ago

@kellogh It also requires a grounding in knowing topics well enough to know when you're getting bullshitted.

The polite term in #AI circles is 'hallucination', but really it's a 5 year old just making stuff up because it doesn't want to disappoint with an "I don't know".

(but 5 year olds deploy the same thing when they're in trouble. So wait for an AI to do that. LOL)

kellogh, 13 days ago

@knowprose yeah, my 3 year old definitely strings together the "most plausible sequence of words"

kellogh, 13 days ago

i also think sam altman is yanking our chain with GPT5. we already know he's manipulative, he's just gone bigger and more public this time. it's either not coming soon, or it's not going to be as promised when it does come

kellogh, 13 days ago

the last week or two there's been some big jumps, in terms of #LLM sustainability.

1. with apple's new models, they bought and paid for all the written works used for training

2. microsoft's #phi3 and also apple's new models are both tiny and capable. in apple's case, tiny enough to fit on a phone

that's what happens when you slow down, things become more sustainable

kellogh, 13 days ago

also meta's llama 3, they kept the model size the same as llama 2, but dramatically increased the performance by using a bigger and higher quality training dataset

smaller models = more environmentally sustainable

redscroll, 13 days ago

@kellogh I'd like to see how the 15T tokens looks like on a 3B model.

anderson_jon, 13 days ago

@kellogh ooo do you have a link to an article or something about apple’s models using paid written work?

kellogh, 13 days ago

@anderson_jon i think i might have either made that up or got the wrong model. i swear it was apple, but tbh it was just something i overheard and didn’t actually verify

https://huggingface.co/apple/OpenELM