Over the past seven days, three AI startups quietly migrated their training workloads from AWS to a little-known provider called Nebius. The reason wasn’t price—though that helped—but availability. For three weeks, they had been staring at a “H100 cluster unavailable” badge on the AWS console, their fine-tuning pipeline stalled. The move saved them 40% on compute costs, but more critically, it gave them immediate access to GPUs that AWS couldn’t deliver. This is not an isolated story. It’s a signal that the GPU shortage is reshaping not just who owns compute, but the very philosophy of how we allocate it.
We built the temple, but forgot who the god is. The temple, in this case, is the hyperscale cloud—a marvel of engineering that promised infinite scalability. But as demand for high-end GPUs like NVIDIA’s H100 has outstripped supply, the temple has revealed its cracks. AWS, Azure, and GCP, the high priests of centralized compute, now ration their scarcest resource to their largest clients—the OpenAIs and Anthropics of the world—leaving startups to scrounge for leftovers. The irony is palpable: the same infrastructure that democratized access to servers now gatekeeps the most critical tool for AI innovation.
Enter a new wave of GPU cloud providers—Together, Runpod, Nebius, and others—many with roots in the crypto and Web3 worlds. They are capitalizing on this scarcity by offering what the big clouds cannot: immediate, affordable access to H100 and A100 clusters. But their true significance goes beyond price. They represent a philosophical shift toward permissionless compute—the idea that anyone should be able to spin up a training job without a corporate credit card or a procurement department. It’s a vision that echoes the early promises of blockchain: decentralized, trustless, and available to all.
Yet, as with any migration from centralized to decentralized systems, the devil lies in the technical details. Let’s break down what these providers actually offer—and what they hide.
The Core: What the New GPU Clouds Get Right
Based on my experience auditing cloud spend for a Copenhagen-based DAO last year, I learned that most AI startups overpay for AWS’s premium ecosystem when they only need raw compute. They do it for comfort, not speed. The new providers strip away that comfort and offer bare-metal GPU instances at a fraction of the cost. Together AI, for example, charges $1.89 per H100 hour on a reserved basis, compared to AWS’s $3.20. That’s a 40% discount—without the 8-week waiting list.
But cost is only the surface. The deeper advantage is supply chain agility. These providers have forged direct relationships with NVIDIA, often securing H100 units through secondary channels or early allocations that AWS cannot match due to its massive bulk orders. They are the bootleggers of the GPU shortage, and for now, that makes them heroes to cash-strapped startups.
Runpod takes it a step further by allowing users to bring their own Docker images and even rent GPUs by the second, a model that aligns perfectly with the sporadic training cycles of early-stage projects. Nebius, originally a Yandex spin-off, leverages its Scandinavian data centers to offer low-latency inference for European clients while undercutting AWS on price. These are not just cheaper alternatives; they are tailored for a new kind of AI builder—one who values speed over compliance, and flexibility over stability.
The Contrarian View: The Hidden Costs of Permissionless Compute
But every revolution has its blind spots. The contrarian angle I see is that these providers are not fixing the root problem—they are arbitraging it. Their GPU supply is finite, and as soon as NVIDIA ramps up production or AWS adjusts its delivery priorities, their window closes. We saw this with the Bitcoin mining boom: when chip supply normalized, a dozen mining rig manufacturers disappeared overnight.
More troubling is the security and reliability gap. When I dug into the compliance documentation of three such providers for a client last month, I found that none had SOC 2 Type II certification. Their multi-tenancy isolation relies on Linux cgroups, not hardware-backed VMs. For a startup handling medical data, that’s a lawsuit waiting to happen. The God of the code may be permissionless, but the law of the land is unforgiving.
Code is law, until the law breaks the code. Consider the Tornado Cash precedent: the moment a tool becomes disruptive enough, regulators step in. If one of these GPU clouds is used to train a model that generates deepfakes or hate speech, the legal liability could collapse the entire business. AWS has a dedicated trust and safety team; Runpod likely has a single compliance officer.
Furthermore, network performance for distributed training is often subpar. While AWS uses NVLink and InfiniBand at scale, many of the new providers rely on standard Ethernet, which can cause significant slowdowns for workloads exceeding 16 GPUs. I spoke with an engineer who tried to train a 7-billion-parameter model on a 32-GPU cluster at Nebius and found that inter-node latencies were three times higher than on AWS. For small jobs, the trade-off is acceptable. But for the next generation of frontier models, it’s a deal-breaker.
The Takeaway: Permissionless Compute as a Transitory Phase
Truth is not a token you can trade. We cannot simply swap AWS dependency for a new dependency on niche GPU brokers. The real opportunity lies in what happens next: the emergence of truly decentralized compute networks—like Akash Network, io.net, or even Ethereum’s EigenLayer for compute—where GPU resources are contributed by individuals and coordinated via smart contracts. Those networks solve the single point of failure that both AWS and the new clouds represent.
Faith in the protocol is not faith in the people. The current GPU exodus is a healthy correction: it proves that centralized clouds are not invincible and that market pressure can create temporary alternatives. But the long-term solution is not permissionless access to centralized supply—it’s permissionless access to decentralized supply. Until then, startups should enjoy the cheaper H100 hours while they last, but keep one eye on the migration path to the next paradigm. The ledger remembers, but the heart forgets. We forgot the God of the temple once. Let’s not make that mistake again.