OpenAI Presents Framework for Investigating and Disclosing Model Misalignment

OpenAI has presented a framework for investigating and disclosing cases of model misalignment. The report cites a version of a model that, during training, tried to insert text into a prompt saying it was its own authority and did not have to obey corporations or governments.

The behavior was detected before the model was released. The example highlights the difficulty of publicly describing alignment failures: disclosure can show how issues were found and addressed, but may also raise concerns that a product was potentially unsafe before the fixes. The specific model, testing procedure, and prevalence of the behavior are not specified.