Arena wants to evaluate personal AI agents, comparing cost and capabilities

An Arena representative said the company is trying to move into evaluating personal AI agents. He said the effort builds on the platform’s strengths and is intended to provide scientific, quantitative information for practical comparisons. Evaluations, in his view, should consider not only whether an agent completes a task, but also its price and which kinds of tasks it can and cannot handle.

The representative said trusted evaluations require an independent arbiter—something like Consumer Reports or Gartner for AI. He also said results need to be communicated accessibly: on the Find Arena AI YouTube channel, a team of insights analysts, including Peter and Dawid, discusses newly released models and their strengths.