artificial_intel

This magazine is from a federated server and may be incomplete. Browse more on the original instance.

LLaMA.cpp: A GPT3-level LLM that can run on your desktop (lemmy.ml)

LLaMA.cpp is a project on GitHub that implements inferencing of a LLaMA model in pure C/C++. The performance is pretty amazing given the limited hardware it can run on (even a Pi, if you have patience), and the author gives an explanation of how that’s even possible (hint: memory bandwidth).

  • All
  • Subscribed
  • Moderated
  • Favorites
  • artificial_intel@lemmy.ml
  • ethstaker
  • DreamBathrooms
  • InstantRegret
  • magazineikmin
  • ngwrru68w68
  • cubers
  • thenastyranch
  • Youngstown
  • rosin
  • slotface
  • cisconetworking
  • mdbf
  • kavyap
  • Durango
  • megavids
  • khanakhh
  • GTA5RPClips
  • anitta
  • osvaldo12
  • everett
  • normalnudes
  • tester
  • tacticalgear
  • provamag3
  • modclub
  • Leos
  • JUstTest
  • lostlight
  • All magazines