This is a crosspost from the official Surge AI blog.
For background on this blog post...
I used to work at Facebook, YouTube, and Twitter. One of the problems I big worked on: what was the right objective function to align our AI systems towards?
Optimizing for watch time at YouTube, for example, led to longer videos for the sake of longer videos, and clicks on videos with racy thumbnails.
Optimizing engagement at Facebook led to low-quality clickbait, and Hooter's appearing as the top search result when you searched for restaurants in Houston.
Optimizing for favorites and replies at Twitter led to toxic content at the top of the feed.
So while watch time, engagement, and replies would always go up – were these really the products we wanted to build? What happened to Facebook's original mission of connecting users with their friends and family? What did "favorites" have to do with being the platform for public conversation at Twitter? A ton of engineering and data science time is spent measuring active users, but why were there no dashboards measuring progress to broader product goals?
So could we figure out a metric that was better tuned to human values and preferences – but also fast, rigorous, and easily measurable? After all, we still need our A/B tests, ML objectives, and OKRs to align the company around.
This question is particularly important today, with all the troubles that social media platforms face, so I wrote up an approach I've often worked on, based on Facebook data. Read it on the Surge AI blog!