Nonlinear Impact of Structural Diversity on User Engagement in Social Networks : Evidence from a Large-scale Field Experiment

Social interactions have become crucial and varied in the digital economy. However, the impact of the structural diversity of social network connections on users' decisions is still not well understood. Accurate measurements and rigorous causal evidence have been lacking, making it difficult to unravel the complex effects. We propose a novel method to measure structural diversity, which is more precise in detecting neighborhood communities and more robust to inter-community connections than traditional methods. Using this new measurement, we conducted a large-scale field experiment on video recommendation and employed advanced econometric tools to causally identify the impact of structural diversity on user engagement, including consumption volume and diversity. Our findings demonstrate that the structural diversity exposed in the recommendation results has an inverted U-shaped impact on user engagement, with optimal structural diversity boosting users' consumption volume and diversity by over 20\%. Exploration of the underlying mechanisms reveals that social utility and privacy concerns are competing drivers of the nonlinear effect. This study reconciles inconsistencies in previous studies on structural diversity and contributes to the literature on social influence, offering far-reaching implications for researchers and practitioners seeking to harness the power of social network information in the digital age