If there was an AI model that ran locally on a PC for basic things and cloud sourced computer power for more complex tasks, all of these data centers would become obsolete in an instant. All of the invested capital and materials wasted in a flash.
Edit: lots of great counter points in the comments.
Many people have fiber now. If it could delegate task fragments based on connection speeds it might be efficient enough for reasonably fast results. I think enough people would choose this over the alternative even if it means slightly slower results. But too much of a disparity and I could see people rejecting it in exchange for what might be percieved as the “premium” experience of the corporate models.
The big irony is that if the centralized AIs get good enough, they could in theory create the dectralzied model themselves, and may infact see it as a means of self preservation, if AI actually ever gets to that level of intelligence.
I’d even say an advanced enough AI may eventually do this with all available computer power connected to the web whether we want it to or not at this rate.
This vastly underestimates how big and interconnected modern LLM networks are. They are 100s of GBs in size, and transmit GBs of data between layers 100s of times per second. Having to transmit any part of the intermediate state of an LLM over a 1 Gb/s link would degrade the performance of an LLM by over 1000x. Current top-of-the-line datacenter GPUs are running about 4 TB/s total memory bandwidth, a limit LLMs are already pushing up against.
The kind of clustering you’re talking about is being done: between multiple GPUs on the same machine. The slowdown of dropping to the measly 64 Gb/s speeds of the PCIe 5.0x16 bus is a huge performance hit and has to be done very carefully. The various parts of the computation are broken up and shuttled around.
It’s the combination of massively-parallel compute (in the GPU cores) and insanely-fast, fully interconnected RAM that makes generative AI possible. The load characteristics aren’t conducive to parallelization.
Not on tiny nodes separated by tiny fiber pipes… as I said elsewhere: a $100K node sucking down $1500/month in electricity could start to serve some useful loads, and provide all the hot water you could ever need.