Anthropic finds preference modeling scales better for aligned language assistants

Anthropic found ranked preference modeling substantially outperformed imitation learning and scaled more favorably, while modest helpful, honest, and harmless interventions improved with model size. Anthropic later stated that this research shaped Claude and documented broad product deployment.