chikim,
@chikim@mastodon.social avatar

Mark Zuckerberg on Llama 3: Apparently Meta stopped training Llama-3-70b before convergence and decided to move onto Llama-4. Meaning they could have kept training and made it smarter! Also llama3-70b multimodal as well as multilingual and bigger context window are coming. https://youtu.be/bc6uFV9CJGg

kellogh,
@kellogh@hachyderm.io avatar

@chikim what a cop out. “sure, our model sucks, but we made it suck so that the next one won’t”. why not just wait until it doesn’t suck…unless you’ve got nothing

chikim,
@chikim@mastodon.social avatar

@kellogh They're on a race with other companies, so I guess it makes sense. You want to move onto next quicly and get better.

kellogh,
@kellogh@hachyderm.io avatar

@chikim idk it just rubs me the wrong way. Why even bring up the next model now? it just sounds like he’s excusing poor performance by stoking hopes

e.g. most cloud companies don’t even talk about what’s next, and if they do, it’s not in the same session as talking about what the current version is

kellogh,
@kellogh@hachyderm.io avatar

@chikim Meta is poorly suited for this game. Their “move fast and break things” philosophy works phenomenally well in the cloud where you can indeed move fast and recover from errors within minutes. But a 3-6 month release cycle is more like Windows, and needs extensive QA up front. IMO it’s the wrong organization to succeed in this game

chikim,
@chikim@mastodon.social avatar

@kellogh Giving AI community free opensource at the same time having to answer the board and investors, I can understand his decision though. If you don't like his style, you can move onto other open source models. :)

kellogh,
@kellogh@hachyderm.io avatar

@chikim well, it’s also not really an open source model, so most hands are probably forced simply because of the terms

  • All
  • Subscribed
  • Moderated
  • Favorites
  • llm
  • rosin
  • cisconetworking
  • thenastyranch
  • magazineikmin
  • hgfsjryuu7
  • DreamBathrooms
  • InstantRegret
  • Youngstown
  • slotface
  • PowerRangers
  • Durango
  • everett
  • kavyap
  • vwfavf
  • anitta
  • modclub
  • ethstaker
  • khanakhh
  • tacticalgear
  • ngwrru68w68
  • osvaldo12
  • mdbf
  • tester
  • cubers
  • Leos
  • GTA5RPClips
  • normalnudes
  • provamag3
  • All magazines