🚨New preprint!
📰The News Comment Gap and Algorithmic Agenda Setting in Online Forums w/ Flora Böwing
We study:
- The differences in comment preferences between journalists and readers
- How comment ranking algorithms prioritise different kinds of comments to display to users
🕵️With data from DER STANDARD, we find, compared to readers, journalists prefer more positive, timely, complex, direct responses, while readers favour comments similar to article content from elite authors.
📈We develop the Feature-Oriented Ranking Utility Metric (FORUM 🥁) to evaluate how well a ranking algorithm (most upvoted, chronological, etc) returns comments by a certain feature (sentiment, readability, etc) compared to best/worst-possible and random baseline.
🛠️FORUM is a versatile, interpretable tool that can be applied to any task of scoring how well a set of items are sorted, far more than just news comments!
🔍With FORUM, we find dramatic differences in how different, even simple, algorithms prioritise different kinds of comments, as measured by their sentiment, readability, lexical diversity, and similarity to the article.
💪We thus argue editors can exercise agenda setting power in comment sections through curational and algorithmic means. The effects on how audience reception to an article is represented by ranking algorithms are larger than those from more traditional measures (e.g. Editor Picks).
🗣️Understanding these preferences and algorithmic consequences is vital in fostering healthy discussion online and of clear interest to journalists presenting their stories. This is especially important given the increasing scrutiny of online speech and how it is hosted/moderated.
👨🏫I presented this work at IC2S2 2024. 📖To find out more, check out the following: 🔗Preprint link: https://arxiv.org/abs/2408.07052 🐍Try FORUM in Python: https://github.com/pgilders/pyforum
🌟Finally, big thanks to Flora, this project came out of her prizewinning work at LSE Methodology!
