ClusterMAX 3 highlights NeoCloud reliability gaps as customers rely heavily on providers

Work on SemiAnalysis’s ClusterMAX 3 took four or five months, with testing focused on B300 GPUs and 800-gigabit networks. Some companies have raised tens or hundreds of millions of dollars and put about 90% of it toward a NeoCloud, making provider choice critical. The evaluation includes performance tests and simulated failures to measure how quickly providers detect problems and recover; some real hardware failures were also induced when providers allowed it.

Top providers completed failure detection and recovery in about 20 minutes. At some others, problems went undetected for hours or days, sometimes until testers alerted the provider, and repairs could take hours or longer. One commentator suggested that high demand and a customer paying about $1 billion a year might affect providers’ priorities, while warning that reliability problems could affect future relationships. NeoClouds opt in to being graded.