I need to have a little lie down after reading about Taalas, the startup that claims to offer a 1000x speed up in LLM inferencing by building dedicated ASICs per model. Well done lads, you’ve invented what? A LUT? A Rom? They claim to be able to churn new models out in two months but their first prototype runs a two year old Llama model badly quantised down to 3 bits. I’m about 90% certain there will be a Theranos outcome for this company. #ai #inference #taalas #theranos #asic https://www.ctol.digital/news/taalas-hc1-review-17000-tokens-per-second-219m-raised-five-risks-every-investor-must-know/
