I think Zuck is right about this, insofar as the comments in the article are concerned.
OpenAI have done a brilliant job of selling the dream - but there will not be one “god model,” - there will be many specialised, smaller models.
You can already see it going that way with new hardware shipping with NPU’s. These workloads are expensive to run and shipping them to your device is a top priority.
ChatGpt already is multiple smaller models. Most guesses peg chatgpt4 as a 8x220 Billion parameter mixture of experts, or 8 220 billion parameter models squished together
I think Zuck is right about this, insofar as the comments in the article are concerned.
OpenAI have done a brilliant job of selling the dream - but there will not be one “god model,” - there will be many specialised, smaller models.
You can already see it going that way with new hardware shipping with NPU’s. These workloads are expensive to run and shipping them to your device is a top priority.
ChatGpt already is multiple smaller models. Most guesses peg chatgpt4 as a 8x220 Billion parameter mixture of experts, or 8 220 billion parameter models squished together