If there was an AI model that ran locally on a PC for basic things and cloud sourced computer power for more complex tasks, all of these data centers would become obsolete in an instant. All of the invested capital and materials wasted in a flash.
Edit: lots of great counter points in the comments.
No, the requirements to run a locally hosted flagship llm is on the order of a Terabyte of vram, like Deepseek v4. The best models from openai and anthropic are likely similar. These are usually loaded into vram on one machine that can serve many users
But if you tried to distribute the compute, if it’s even possible, would take like a hundred high end gaming computers and process at a glacial pace. It just doesn’t make sense.
It makes more sense for an organization to purchase their own hardware and distribute resources somehow, like for a University, but consumer hardware won’t really cut it.
You could use smaller models, but their performance scales with size so they are much dumber than the flagship models.
Not really. A big problem is the energy usage, which would not go away.
Distributed computing is a real thing though. Check out GIMPS and Folding@home if this interests you. The first is looking for extremely large prime numbers. The second is to learn about proteins. I believe amateurs get more benefit from it. I remember a service to help jailbreak your 3ds used distributed computing. Also there are some folks who search for Minecraft world generation seeds with specific properties using distributed computing.
For the local on pc part yes, if you have a decent pc. That is why they want to forbid open source weights models.
For the crowdsourced part not really. You need fast communication between your nodes for this to work, and fast here means really fast. To the point where ram to gpu on the same machine is not fast enough, so network is out of the question here.
This for the technical reasoning.
Companies also just don’t want people to own anything, they want everyone to pay a subscription for everything forever.
If they would just install $100K compute nodes in every home, that should be enough local high bandwidth to do the job. It will also run about $1000-$1500 per month in electricity costs, but, hey, it’s about time to upgrade that grid to be able to quick charge EVs whenever we want, too, isn’t it? You might even try using the compute node to make hot water and heat your home in the winter - gonna be kind of a challenge to muffle the fan noise though.
Many people have fiber now. If it could delegate task fragments based on connection speeds it might be efficient enough for reasonably fast results. I think enough people would choose this over the alternative even if it means slightly slower results. But too much of a disparity and I could see people rejecting it in exchange for what might be percieved as the “premium” experience of the corporate models.
The big irony is that if the centralized AIs get good enough, they could in theory create the dectralzied model themselves, and may infact see it as a means of self preservation, if AI actually ever gets to that level of intelligence.
I’d even say an advanced enough AI may eventually do this with all available computer power connected to the web whether we want it to or not at this rate.
This vastly underestimates how big and interconnected modern LLM networks are. They are 100s of GBs in size, and transmit GBs of data between layers 100s of times per second. Having to transmit any part of the intermediate state of an LLM over a 1 Gb/s link would degrade the performance of an LLM by over 1000x. Current top-of-the-line datacenter GPUs are running about 4 TB/s total memory bandwidth, a limit LLMs are already pushing up against.
The kind of clustering you’re talking about is being done: between multiple GPUs on the same machine. The slowdown of dropping to the measly 64 Gb/s speeds of the PCIe 5.0x16 bus is a huge performance hit and has to be done very carefully. The various parts of the computation are broken up and shuttled around.
It’s the combination of massively-parallel compute (in the GPU cores) and insanely-fast, fully interconnected RAM that makes generative AI possible. The load characteristics aren’t conducive to parallelization.
The load characteristics aren’t conducive to parallelization.
Not on tiny nodes separated by tiny fiber pipes… as I said elsewhere: a $100K node sucking down $1500/month in electricity could start to serve some useful loads, and provide all the hot water you could ever need.
The data centers still have more compute, they have more money and regular people are being priced out of hardware because of datacenter demand, and they already had more to begin with. The question is whether the market demand for that compute is effectively unlimited, or if there’s a point of diminishing returns. If the former, then no amount of crowdsourcing would be able to make datacenters obsolete because home computers cannot satisfy the demand on their own. If the latter, then yeah it is possible.
If we get to a point where anything you’d really want AI for can be achieved with a smaller model and more powerful stuff is niche and rarely needed, then data center compute may become obsolete. Local models are already very useful and good enough is good enough.
It’s not very energy-efficient. That’s one of the reasons supercomputer clusters and datacenters exist despite all the cooling requirements.
The data centers would not be obsolete as I imagine for most people running locally is not gonna cut it for 99% of stuff they use AI for and expect from it.
Even peeps with absurd rgb gaming rigs are likely not gonna wanna drop down settings and fps so they can have a local llm helping them cheat at pretending they are a soldier or whatever.
People could have dedicated AI servers that they can access with their entertainment PC.
This is of course very hypothetical.
If people in general can’t be arsed self hosting media streamers, email and social media then thinking they are gonna start running novel AI datacentres in their cupboard seems somewhat optimistic.
Even if something like a GDX Sparx could be gotten cheaply you still need to pay the electric bill to run it and be a system administrator so likely just ends up costing a load to buy and run and you still have to pay for datacentre access for large models.
There’s also a bit of an issue with hardware at the moment which is why I’m pursuing your vision on a n100 minipc.
I use Jan on my workstation which is in the kinda area, it comes with a little local model and you can plugin your own models and cloud api’s.
It’s a matter of degrees, one in a million consumers might be enough of an “enthusiast” to spend the money to setup a personal server. Even if I had the money ($100K) for a “powerful enough” server, the market is changing so fast it’s definitely rent-not-buy territory for anybody with any sense.
With how much money AI-using companies are bleeding and the AI-providing companies are still not profitable I doubt that crowd sourcing would work. People would just notice that their PC is being hammered day and night and opt out eventually.
Hypothetically, people could have an AI dedicated server, and when the AI is inactive for local tasks it helps the AI hivemind.
Too slow and I guess most people don’t have good enough computers for that.
Also what is the incentive? SETI@home was fun. AI is not fun at all.
A dedicated server that gets anywhere close to “frontier model” performance is still about 100x as expensive and power hungry as an average home PC.
There are already AIs you can run on your computer or even your cell phone, they are just really really specialized or dumb.
Even if you could magically network every desktop, laptop, workstation and cell phone in the world and run them all full power 24 hours a day doing nothing but powering this network, you are probably an order of magnitude or more behind even current data center capacity.
No it would be abused to hell and back. Or be unreliable.
