The Unintended Consequences of Preference Tuning LLMs

Invited industry talk on how RLHF-style alignment creates disparities across English dialects and global opinions, from his ACL 2024 paper 'Unintended Impacts of LLM Alignment on Global Representation'.