2026 Enterprise AI Infrastructure in China: Current State and the GPU Procurement Boom

From Cloud to Server Room: The Return of Enterprise AI Infrastructure

At the end of 2025, a manufacturing client in Beijing with annual revenue of RMB 300 million came to us. They were preparing to migrate their AI quality-inspection model — until then running in the cloud — back on-premises. “The monthly inference bill costs more than the combined salaries of our three engineers,” said Lao Zhang, the technical lead, counting on his fingers, “and GPU utilization is still under 30% — the waste hurts.” This scenario has repeated itself over the past year. Based on the more than 30 manufacturing, healthcare, and financial clients we have worked with, 60% of enterprises were evaluating or had already begun building local AI clusters in 2025 — a figure that was still under 25% in 2024. The forces behind this wave are clear: LLM API costs remain persistently high, open-source models such as DeepSeek are gradually approaching GPT-4 performance, and market prices for NVIDIA H100/H200 began to soften in the second half of 2025. Enterprises’ compute anxiety has shifted from “can we afford it” to “how do we make good use of it once we buy.” One notable change: purchasing decisions are moving down from the CTO’s office to business unit leaders — factory floor supervisors raise the requirement, headquarters CIO approves it. This pattern is becoming the new normal.

DGX Spark, Huawei Ascend, and Domestic GPUs: The Reality of Three Paths

On hardware selection, the Chinese market in 2026 shows a clear three-way split. DGX Spark, NVIDIA’s edge computing product launched in 2025, centers on single-node inference performance of 1 PetaFLOPS and suits small and mid-sized enterprises that need rapid deployment and have limited operations expertise. In Shunyi we met a 200-person e-commerce company running a product recommendation model on two DGX Sparks; response latency fell from 800 milliseconds in the cloud to 120 milliseconds, and click-through conversion rose by 12%. But the problem is very real: a single DGX Spark still sells for RMB 150,000–200,000, and once after-sales maintenance and power costs are added, the upfront burden on an SME is considerable. Huawei Ascend 910B/C, by contrast, holds a firm leading position in domestic large-model training and government-related scenarios. In 2025, a municipality’s smart city project purchased 500 Ascend servers in a single order, for a total of more than RMB 200 million. On technical specs, the Ascend 910B delivers FP16 compute of 256 TFLOPS, narrowing the gap with the H100, and its optimization under the MindSpore framework is already quite mature. In actual deployment, however, the issue enterprises most often raise is the software ecosystem — Ascend-adapted versions of many open-source models require additional development cycles, which slows projects down to a degree.

An AI Decision Framework for SMEs: From “Should We Buy?” to “How Much, and How Do We Sustain It?”

The question we were asked most over the past year: “At our scale, should we really build our own GPU cluster?” My advice is to break that question into three dimensions:

1. Compute density of the business scenario. If the AI workload is online inference running 24/7 (a customer service chatbot or real-time risk control, for example), on-premises deployment delivers significant cost and latency gains; but if it is a data analysis job that runs once a day, on-demand cloud calls may be more economical.
2. The team’s capacity to absorb the technology. A GPU cluster does not work the moment you buy it. Driver debugging, multi-GPU interconnect, model deployment, monitoring and alerting — this operations stack requires at least one or two engineers with Linux system administration experience. Without that staffing in reserve, actual utilization after purchase is often dismal.
3. Data compliance and privacy boundaries. Sensitive data in healthcare, finance, and manufacturing is often subject to regulatory constraints, and on-premises deployment is a hard requirement for MLPS 2.0 (China’s Multi-Level Protection Scheme) compliance. In such scenarios, building your own cluster is not only a matter of economics but of compliance.

A practical reference point is the “three-year TCO model”: estimate three years of cloud API costs, then compare that with the purchase cost of a local cluster over the same period, plus electricity and operations staffing. If the former exceeds the latter by 1.5 times, on-premises deployment deserves serious consideration. We have run this calculation for clients many times, and for many manufacturing plants the quality-inspection AI recovered its hardware cost within three months of moving on-premises. That said, one point must be made: building a GPU cluster is not simply a matter of buying a few servers. Network architecture, storage I/O, and cooling retrofits — these supporting elements often account for 20%–30% of total investment. If the enterprise lacks in-house IT infrastructure experience, we recommend prioritizing a turnkey delivery solution rather than purchasing hardware piecemeal and having a systems integrator assemble it. One of our clients in Wangjing learned this the hard way — they bought top-tier servers but chose a gigabit network for storage, which made training data loading a bottleneck and forced a second round of rework.