Much of the public discussion of AI infrastructure still centres on training — the construction of enormous clusters dedicated to building ever-larger models. Yet a growing body of industry analysis suggests that inference, the process of actually running trained models in production, already accounts for the large majority of AI compute cycles. That shift carries real implications for where, and how, AI infrastructure should be built.
Training and Inference Are Different Infrastructure Problems
Training workloads benefit from extreme concentration: thousands of accelerators in a single facility, tightly coupled by ultra-low-latency networking, working in synchrony on a single model for weeks or months. Inference workloads, by contrast, are typically smaller in per-request compute terms, far more numerous, and frequently latency-sensitive in ways that favour proximity to end users rather than proximity to the cheapest power.
As AI moves further into consumer applications, enterprise software, and real-time decision systems, the geographic logic of where inference capacity should sit increasingly diverges from the logic that has dominated training cluster siting — namely, going wherever power and land are most available, regardless of distance from population centres.
What This Could Mean for Facility Geography
If inference continues to represent the dominant share of AI compute demand, it is likely to support a more distributed data center landscape than the training-led narrative alone would suggest — a larger number of moderately sized, latency-optimised facilities positioned closer to where AI applications are actually used, complementing rather than replacing concentrated training campuses.
- Latency-sensitive applications — real-time recommendation, conversational AI, autonomous systems — benefit from inference capacity located closer to end users
- Regulatory and data residency requirements in many jurisdictions favour in-region inference capacity, independent of pure latency considerations
- Inference workloads, while still GPU-intensive, generally impose less extreme per-rack power density than frontier training clusters, which can open up a broader set of viable sites
Training concentrates compute where power is cheapest. Inference is increasingly likely to distribute compute to where users actually are.
An Important Caveat: This Is a Developing Picture, Not a Settled One
It is worth treating this shift with appropriate caution rather than certainty. Inference workload characteristics continue to evolve quickly — more complex reasoning models and AI agents are themselves becoming more compute-intensive per request, which could partially offset the geographic distribution effect by making concentrated inference capacity more economically attractive in some cases. The likely outcome is not a simple substitution of distributed inference for concentrated training, but a more layered infrastructure landscape with both models coexisting and serving different requirements.
What Developers Are Already Doing About It
Several leading data center operators are already structuring their portfolios to reflect a layered, rather than single-model, infrastructure future. This typically involves maintaining a core of large, power-secured campuses for training and the most compute-intensive inference workloads, alongside an expanding network of smaller, well-connected facilities in or near major demand centres for latency-sensitive inference and regional data residency requirements. The two facility types require different design priorities — the large campus optimises for power and density at almost any cost, while the distributed facility optimises for connectivity, proximity, and often a lower but still meaningful power density.
This bifurcated strategy also has portfolio diversification benefits independent of the underlying technology trend: a portfolio spread across both facility types is less exposed to any single shift in AI workload patterns than one concentrated entirely in either large training campuses or smaller inference-oriented sites.
Connectivity Becomes the Differentiator for Distributed Capacity
As inference-oriented capacity becomes more distributed, the quality and redundancy of network connectivity at a given site becomes a more important differentiator than it has historically been for data center site selection. Proximity to internet exchanges, diverse fiber routes, and low-latency connections to major population centres increasingly matter as much for a distributed inference facility as raw power availability matters for a training campus.
Strategic Implications for Developers and Investors
For developers, this argues for site selection and product strategy that does not assume every AI facility needs to chase the largest possible power allocation in the most remote available location. Smaller, well-connected sites closer to demand centres may be increasingly valuable for inference-oriented capacity, even as large training campuses continue to anchor the high end of the market.
DATAPERT advises clients across both ends of this spectrum — from large-scale hyperscale data center programmes to more distributed digital infrastructure strategies. Explore our investment intelligence capabilities or start a project to discuss your specific infrastructure strategy.
