Clockwork.io raises $31M to keep AI workloads running through GPU failures

Clockwork.io has raised $31 million in new funding as demand grows for software that can keep expensive AI workloads running when GPUs, network links, or servers fail.

The round was co-led by Premji Invest, Wing Venture Capital, and Seligman Ventures, with participation from existing investors NEA and e& Capital. The new financing brings Palo Alto-based AI infrastructure startup’s total funding to $73 million.

The raise comes with a stronger proof point than funding alone. LinkedIn has deployed Clockwork.io’s network fault-tolerance software across its AI infrastructure fleet, where the company says it is preventing tens of thousands of GPU-hours of downtime each month. Together AI is bringing Clockwork.io’s TorchPass technology to customers through its GPU Clusters, and WhiteFiber is increasing its use of the company’s software across its GPU infrastructure.

That matters as AI training jobs grow larger and more expensive. A distributed workload may stretch across thousands of GPUs, and a single failed component can interrupt work across the cluster. Meta previously reported unexpected interruptions averaging roughly once every three hours during a 54-day period of Llama 3 training on 16,384 GPUs.

A failure does not just create a technical headache. It can leave costly hardware idle, force jobs to repeat completed work, and stretch training times.

Clockwork.io is trying to reduce that waste by treating infrastructure failures as something AI systems should work through rather than stop for.

Keeping AI jobs alive when hardware fails

Clockwork.io’s software sits between AI workloads and the infrastructure running them. Its LinkPass product reroutes traffic when a network link fails. TorchPass shifts work away from failing GPUs so jobs can continue without restarting from an earlier checkpoint.

The company introduced new TorchPass features alongside the funding announcement. One can capture the state of a distributed training job across multiple nodes, giving platform teams a way to restore the workload after a larger failure without changing the training code. Another takes application checkpoints in the background, which can shorten recovery times and help reinforcement-learning systems move updated model weights between training and inference systems more quickly.

New TorchPass innovations preserve AI workload progress without code changes and speed reinforcement learning. (Credit: Clockwork.io)

“Failures are inevitable at AI scale. Losing hours of useful work to them should not be,” Clockwork.io CEO Suresh Vasudevan said. “Fault tolerance is a goodput multiplier: it keeps GPUs doing useful work instead of waiting for recovery or repeating work already done.”

LinkedIn offers a glimpse at the economics behind that argument.

“At AI infrastructure scale, a single network issue should never sideline healthy GPUs or interrupt running workloads,” said Raghu Hiremagalur, LinkedIn’s SVP and CTO of Infrastructure. He said Clockwork.io’s technology now helps the company avoid tens of thousands of GPU-hours of downtime each month.

Clockwork.io already works with companies including Wells Fargo and Uber, and it partners with cloud and AI infrastructure providers such as Nebius and NScale.

The company plans to use the new capital to roll out its software across more training, inference, and reinforcement-learning workloads, win more enterprise customers, and increase distribution through cloud and neocloud partners.

That puts Clockwork.io inside one of AI infrastructure’s less glamorous but increasingly expensive problems. The industry has spent years racing to acquire more GPUs. The next battle is making sure those GPUs spend more time doing useful work.