POSTED ON JANUARY 14, 2026 TO ML Applications, Video Engineering
Improving AI Recommendations on Facebook Reels Through User Surveys
- Personalized video recommendations on Facebook Reels have been enhanced by directly incorporating user feedback, moving beyond traditional metrics like likes and watch time.
- A new User True Interest Survey (UTIS) model now helps surface more niche, high-quality content, boosting engagement, retention, and user satisfaction.
- The focus is on enhancing personalization, addressing challenges such as sparse user data and bias, and exploring advanced AI techniques for smarter and more diverse recommendations.
- A paper, “Improve the Personalization of Large-Scale Ranking Systems by Integrating User Survey Feedback,” provides comprehensive details on this work.
Providing personalized video recommendations presents a significant challenge for maintaining user satisfaction and long-term engagement on large social platforms. Facebook Reels has addressed this by prioritizing “interest matching,” which ensures that displayed content genuinely aligns with individual user preferences. Through the integration of large-scale user surveys and advanced machine learning, the platform can now better understand and model what users truly care about, leading to notable improvements in recommendation quality and overall user satisfaction.
Why True Interest Matters
Traditional recommendation systems frequently depend on engagement signals like likes, shares, and watch time, or use heuristics to infer user interests. However, these signals can be imprecise and may not fully reflect the subtle aspects of what users genuinely care about or wish to see. Models trained solely on these signals often suggest content that offers high short-term value, as measured by watch time and engagement, but may not capture the true interests crucial for long-term product utility. To overcome this, a more direct method was needed to gauge user perception of content relevance. Research indicates that effective interest matching extends beyond mere topic alignment, incorporating elements such as audio, production style, mood, and motivation. By accurately capturing these various dimensions, recommendations can feel more relevant and personalized, encouraging users to engage with the app more consistently.
Recommendation systems are usually optimized using user interactions like watch time, likes, and shares. However, integrating user perception feedback, such as interest match and novelty, can significantly enhance relevance, quality, and the overall content ecosystem.
How We Measured User Perception
To validate this approach, large-scale, randomized surveys were implemented within the video feed, posing the question, “How well does this video match your interests?” These surveys were rolled out across Facebook Reels and other video platforms, gathering thousands of in-context responses daily. Findings indicated that prior interest heuristics achieved only 48.3% precision in identifying true interests, underscoring the necessity for a more robust measurement framework. By weighting responses to account for sampling and nonresponse bias, a comprehensive dataset was constructed that accurately reflects genuine user preferences, shifting from implicit engagement signals to direct, real-time user feedback.
Framework: User True Interest Survey (UTIS) Model
Each day, a random selection of user viewing sessions on the platform displays a single-question survey: “To what extent does this video match your interests?” on a 1-5 scale. This survey collects real-time feedback from users regarding recently viewed content. The platform’s primary candidate ranking model is a large multi-task, multi-label model. A lightweight UTIS alignment model layer was trained using the collected user survey responses, leveraging existing predictions from the main model as input features. Survey responses were binarized for simplified modeling and to reduce variance. Additionally, new features were engineered to capture user behavior, content attributes, and interest signals, with the objective of optimizing the prediction of users’ interest-matching extent. The UTIS model generates a probability that a user will be satisfied with a video and is designed for interpretability, allowing for an understanding of the factors influencing users’ interest matching experience.
User perception feedback gathered through surveys is often sparse, but this feedback can be generalized across large-scale recommendation systems using a novel “Perception Layer” architecture that incorporates existing event predictions as additional features.
Integrating the UTIS Model in the Main Ranking System
Several applications of the UTIS model have been tested and implemented within the ranking funnel, all demonstrating successful improvements in tier 0 user retention metrics:
- Late Stage Ranking (LSR): UTIS operates in parallel with the LSR model, contributing an additional input feature to the final value formula. This enables fine-tuning of the final ranking stage to incorporate true interests while balancing other considerations.
- Early Stage Ranking (Retrieval): UTIS is utilized to reconstruct users’ true interest profiles by aggregating survey data to predict affinity for any given user-video pair. This allows for re-ranking of user interest profiles and sourcing more candidates relevant to users’ genuine interests. Additionally, large sequences from user-to-item retrieval models are aligned using knowledge distillation-based objectives trained on UTIS predictions from LSR as labels.
The UTIS model score now serves as one of the inputs to the ranking system. Videos predicted to be of high interest receive a moderate boost, while those with low predicted interest are demoted. This approach has resulted in:
- Increased delivery of high-quality, niche content.
- A reduction in low-quality, generic popularity-based recommendations.
- Improvements in like, share, and follow rates.
- Improved user engagement and retention metrics.
Since the implementation of this approach, robust offline and online performance has been observed:
- Offline Performance: The UTIS model showed improved accuracy and reliability compared to the heuristic rule baseline. Accuracy rose from 59.5% to 71.5%, precision improved from 48.3% to 63.2%, and recall increased from 45.4% to 66.1%. These gains demonstrate the model’s capability in accurately identifying users’ interest preferences.
- Online Performance: Large-scale A/B testing involving over 10 million users confirmed these improvements in real-world scenarios. The UTIS model consistently outperformed the baseline, leading to higher user engagement and retention. Specifically, a +5.4% increase in high survey ratings was observed, alongside a -6.84% reduction in low survey ratings, a +5.2% boost in total user engagement, and a -0.34% decrease in integrity violations. These results underscore the model’s effectiveness in enhancing user experience and connecting users with relevant interests.
Future Work for Interest Recommendations
The integration of survey-based measurement with machine learning is fostering a more engaging and personalized experience, delivering content on Facebook Reels that feels genuinely tailored to each user and encourages repeat visits. While survey-driven modeling has already enhanced recommendations, significant opportunities for further improvement exist. These include better serving users with limited engagement histories, mitigating bias in survey sampling and delivery, further personalizing recommendations for diverse user groups, and enhancing recommendation diversity. To tackle these challenges and continue advancing relevance and quality, advanced modeling techniques, such as large language models and more granular user representations, are also being explored.
Read the Paper
Improve the Personalization of Large-Scale Ranking Systems by Integrating User Survey Feedback
