It seems like there will be a collapse in building data centers, to the point that a large amount of the newly built ones will be reclaimed for other things. There is a very large group of people who are against building new data centers because they hate AI (and any pollution related arguments these people make are in support of their primary concern, being AI). But on the flip side of that, even people that I know that go out of their way to use AI, mostly seem interested in building their own AI rigs and running everything locally. I know that's what I do; I almost never use any hosted AI services these days, but I'm using more tokens than ever. The demand for data centers from people who actually like AI even seems to be plummeting.
We're already at the point where a few months worth of an SMB's token usage from the SaaS LLM providers costs is equivalent to the cost of installing on-prem infrastructure capable of running cutting-edge open-weight models at scale. At my company, we recently installed a server with an array of Gaudi 2 cards on our server rack, and have set up an Open WebUI frontend to expose an LLM connected to all of our internal resources to our staff. The total cost was about $20k, which we'd easily eat up with six months worth of equivalent Claude usage.
We'd originally set this up to be able to locally run larger models, in the 300-400 billion parameter range, but the rate of improvement of open-weight models has been so fast that, coupled with extensive custom skill creation, we're now getting similar results out of Qwen3.8 27B to what we were getting out of Qwen 3.5 297B when we started out with the project, which frees enough memory to allow 20-25 users to have 256K context concurrently. Both the hardware, the software, and the models are improving at an accelerating rate.
Investing in data centers to support SaaS LLM providers today feels a bit like investing in mainframes and minicomputers in the late '70s, with a massive paradigm shift lurking right around the corner.
Actually, it's probably already closer to the early '80s, given that purpose-built local AI workstations are already available at price points lower than the inflation-adjusted initial price of the original IBM PC.
While I cautiously circle the AI programming concept, one thing close to my mind has definitely been whether the ultimately cloud-tied nature of it is what makes me much more stand offish to it.
I am GPU poor and have a piece of crap rx570 8GB but still get ok performance out of qwen3.5-9B-4bit and various fine tuned versions. Plan on finally buying a decent gpu like a amd 7900 xtx. I still use claude for coding primarily but do research locally. What everyone has said about qwen3.8 though is making me feel I really need to invest in a decent rig.
Also running Qwen 3.8 27b, on a 7900 XTX. I've been able to comfortably do a large majority of my programming locally since about Qwen 3.5.
I also use pi.dev as my agent harness, and ollama as my inference engine. Though I've been considering switching out the latter for something else, since the main advantage of ollama is the ease of switching models, which I don't really do often anymore.
You don’t get to 25 or 30 trillion total addressable market by providing AI for the people who like AI. The whole plan is to replace all white collar work with AI. This requires more data centers. The people who benefit the most from this will see the least of the negative effects.
You don’t get to 25 or 30 trillion total addressable market, full stop. It's just a straight up pipe dream. If everyones jobs are automated away, there won't be any customers for the products and services provided by the companies that are fully automated by AI.
I don’t think you do either, but that doesn’t mean these people aren’t going to build a ton of data centers on that narrative.
Actually I think the AI revolution is going to have a worse outcome. It will kill off the ladder to a better life by eliminating entry level white collar work. This won’t be valuable enough to generate a UBI, and there won’t be some whale company to get it from.
Current day mid-level and up white collar workers will just adopt AI into their work, and likely benefit. In 10-15 years there won’t be replacements but maybe then the AI can run these companies anyways. In the meantime companies will adjust their goods and services to target the increasingly wealthy but shrinking upper class or the growing lower class.
What? (Edit: I think you misread "in support of their primary concern, being AI" as "in support of AI"? People who complain about pollution are usually mostly concerned about AI, and they use arguments of pollution to strengthen their argument against AI related things.)
> And even if you do inference at home you are not training the models.
No, and training does use a large amount of energy on a large amount of hardware. But:
- That sort of workload doesn't really require many distributed data centers, only a few powerful ones. I believe training is also getting cheaper for the achieving higher levels of capabilities, but I don't think the efficiency advancement has been as dramatic as it has been for inference.
- I believe we are really hitting a wall of diminishing returns, especially at the high end of large models. There is lower demand for training, because models are good enough to have a reasonably long shelf life at this point. Year+ old models that were SOTA in their time are still useful today. The demand for training is going down.
> Moreover, most people certainly are not doing inference locally.
Maybe not, but I think most people who go out of their way to use LLMs because they find them useful for their work actually are. Though most inference is probably from people who do it accidentally (as a part of a search result or something) or students who don't have the resources to do it themselves. But basically every software engineer that I personally know that uses AI for programming is either running their inference on their own hardware, or is talking about building a rig to do it.
It’s wild to accuse people who are anti pollution of being anti ai, look how fucked the climate is, people don’t want to see it more fucked , is that hard to understand ?
It's not hard to understand. What's hard to understand is why when I bring up how cars are literally orders of magnitude larger polluters and how we need to eradicate personal car use, people suddenly stop caring about pollution, and continue to beat the anti-AI drum.
See: they were anti-AI the whole time, and pollution was just an attempt at supporting their anti-AI argument.
AI doesn't rub rubber on asphalt at 70 mph. Did you know that?
Electric cars and ICE cars both use more watts than AI. If you're concern is energy consumption, cars are worse. If your concern is pollution, cars are worse. Do you care about pollution and energy consumption, or do you honestly just hate AI? It's okay if you just hate AI, but be honest about it.
Back to the original objection from me, it's fine to object to data centers of being concerned for the environment. You can even like AI and still realize that we need to make sure the data centers are being powered by low / zero emission technologies.
Because as you obviously can tell, it's totally ridiculous to suggest people give up their primary mode of transport so people can have data centers instead.
The people who grow your food aren't going to be doing that on a bus. The people who build and run data centers need cars to do it.
I never suggested people should abandon cars for AI, I think people should abandon their primary mode of transportation for the same arguments that people usually make against AI. Cars have the same negative effects times 10 per person who uses it, and a lot moe people are addicted to cars than AI. And that's before we mention things like the massive amount of straight up violence cars induce that is swept under the rug as inevitable.
I'm not sure what you think farmers need personal cars for? Tractors and trains seem like they'd do literally anything you could imagine a farmer "needing" a car for?
Not sure if you're serious but maybe checkout some YT videos on farming to see what's involved...
Tractors are crazy slow, and it's obviously stupidly inefficient to have trains running into every rural area just for the off chance a farmer needs to go pickup some hay, which by the way would usually go onto the back of a lorry.
