Open and closed AI labs are largely scaling the same training recipe

An assessment of current model development says there is little difference between OpenAI’s reinforcement-learning stack, Chinese open-source efforts and American open-source projects. The approach first scaled pre-training and model size and is now scaling reinforcement learning (RL).

The view is that labs will continue scaling RL, on the bet that it will produce higher levels of intelligence and enable more economically useful tasks. Continual learning is also identified as an important priority.