Social media listening is the practice of collecting and analyzing user comments on social media in an effort to assess consumer sentiment surrounding a particular brand. In this research, we contribute to the social media listening research and propose a brand favorability measure, based on large-scale social media data, that can be used for benchmarking against other brands. In developing our brand favorability measure, we draw from research related to traditional survey methodology. We account for directional biases exhibited by social media posters (i.e., some social media users are generally more positive while others are generally more negative), consider how this bias affects social media metrics of sentiment, and develop a method designed for large scale social media data that provides an adjusted brand favorability measure that is correlated with traditional survey-based measures used by brands. For our analysis, we collect and examine Facebook data for more than 3000 brands and the 170 million unique users that interact with those brands via their Facebook brand page. Our data set is large and contain 6.68 billion likes and full text for 947.6 million posted user comments, creating challenges for any modeling efforts. We find that for many brands, average sentiment metrics deviate from the brand's underlying favorability, measured either by a traditional survey or our proposed method. We explore these deviations and examine how observable factors related to the brand community (e.g., number of followers, number of comments and likes, variance in sentiment), brand traits (e.g., industry sector, size of firm, general popularity), and brand activity (e.g., posting behavior, news mentions) can affect the reliability of average sentiment as a brand favorability measure. This provides brand managers the ability to identify situations where their social media sentiment metrics are likely to deviate from and misstate underlying consumer opinions toward the brand. Empirically, we find that smaller brand communities with limited opinion variance are associated with average sentiment metrics that overstate brand favorability
