has anyone looked into how much a #groq chip/board costs? i submitted an inquiry, but i don’t think they’re going to get back to me. curious how many digits are on the price tag per unit… #LLMs#LLM#AI#GPU#hardware
Groq sounds exciting with it's LPU (Language Processing Unit) as a faster and more efficient alternative to GPU's for LLM's. However difficult to establish the viability with only a website, whitepaper and online demo. Time will tell. https://groq.com/ #AI#LPU#LLM#Groq
Now that #NeuralNetworks have had repeated big successes over the last 15 years, we are starting to look for better ways to implement them. Some new ones for me:
#Groq notes that NNs are bandwidth-bound from memory to GPU. They built a LPU specifically designed for #LLMs https://groq.com/
A wild one — exchange the silicon for moving parts, good old Newtonian physics. Dramatic drop in power utilization and maps to most NN architectures (h/t @FMarquardtGroup)