AI training can be centralised in remote locations, but proximity and latency are fundamental to end users’ experience when AI is ‘everywhere’ – many neoclouds are not ready for the shift
Dr Thomas King, CEO of DE-CIX, warns neoclouds are heading for trouble unless they take prompt action. DE-CIX (for Deutscher Commercial Internet Exchange) describes itself as the world’s leading internet exchange point (IPX). It was founded in 1995 with headquarters in Frankfurt and enables thousands of ISPs, content providers and enterprise networks to interconnect directly to exchange data traffic (peering) and access cloud services. This makes global internet routing faster and more efficient.
Neoclouds offer GPUs on demand for AI and high performance computing whereas the hyperscalers like AWS, Azure and Google Cloud offer generic IT services. The thinking is the purpose-built architecture is faster and cheaper with superior performance.
This holds good until AI reaches the tipping point when most of AI workloads shift from training models to inference. At that point, today’s AI usage will pale into insignificance as AI proliferates – and proximity and low latency assume paramount importance.
Shifting workloads
Currently only 16% of people around the world use AI, on average although obviously it is very much higher in some places as illustrated by a survey published by the OECD of AI usage 14 member countries last December.
It found an average of almost 40% of those populations were active users of GenAI, with adoption highest among 18-35 year olds.
Dr King’s contention is the shift to inference will happen faster than some think. According to data centre specialist JLL, inference will overtake training workloads long before Gartner’s forecast of 2029. By 2027, JLL anticipates that inference workloads will have already caught up, and by 2030 two thirds of all AI workloads will be inference based.
The game changes, enter CDNs
This changes the game because AI training can be centralised in remote locations where power and land are plentiful “and data can be trucked in”. On the other hand, inference needs real-time interaction with end users, whether they are devices, individuals or enterprises. Hence proximity and low latency become critical to the experience expected by those users.
And where have we come this need for promixity and low latency before? For streaming video content on demand, provided by content delivery networks (CDNs) such as Akamai, Cloudflare, Fastly and G-Core Labs.
Dr King is confident they will enter the inference market, applying their expertise from content distribution – if they have not already. This means more competition for the neoclouds in addition the hyperscalers.
How to stay in the game
He states, “For neoclouds to ensure that they can get a piece of the action in the coming inference wave, they need to start thinking like network operators” as for the first time, connectivity assumes a dominant role in neocloud architecture. Yet Omdia Informa’s TechTarget research, published in April, shows that more than 50% of neoclouds do not yet use peering exchanges and 20% still rely on a single transit provider, risking a single point of failure.
The research points out that one of few aggregation points for neoclouds so far is DE-CIX Frankfurt, which explains Dr King’s insights and interest. He claims DE-CIX is evolving into a Network-as-a-Service provider for cloud and AI connectivity to allow neoclouds to evolve their networks and business cases too.
He points out that Neocloud providers like CoreWeave and Nebius are available across multiple locations in Europe and the US on the DE-CIX platform. The connectivity choices made by other pose the risk of their network becoming the bottleneck to performance, efficiency, and revenue growth as the inference wave builds, he says.
Dr King reckons the inference market is big enough for hyperscalers, CDNs and neoclouds to capture a good enough share. He quotes Precedence Research, which in April forecast the global market size of AI Inference-as-a-Service will grow at more than 25% CAGR to a volume of close to $200 billion by 2035.
Read Dr King’s blog in full here.
