kellogh, 18 days ago

@paninid @AnnemarieBridy latency touches every part of every business case

• environmental impact is less
• cost is less
• interactive apps see drastically better user experience
• some applications weren’t possible but are enabled by lower latency

if you don’t understand the user experience impact, try using Groq at 500 tokens/s https://groq.com/