OpenAI experiment connects Codex to a physical robot for painting

An OpenAI robotics-team experiment connected a camera-equipped physical robot to Codex, extending models’ abilities in painting and computer-use tasks to real-world drawing. Early robot paintings took roughly one to two hours, depending on the method used.

Instead of repeatedly processing an image after every small movement, the system writes an action plan in code, executes it, and checks images every second or few seconds to make adjustments. Early demonstrations used 512p images; lower resolution was considered sufficient for longer action plans, while faster control loops are expected to depend more on image resolution.

The exact robotics methods are still being developed, and the current setup is described as slow and potentially expensive. The experiment points to a likely combination of cloud intelligence, local models and specialized interfaces, while cheaper physical devices could help drive a boom in DIY robotics.