Do we have good online reputation systems?

Abstract: Reputation systems are crucial to online platforms’ health. They are prevalent across online marketplaces and social media platforms either visibly (e.g., as star ratings and badges) or invisibly as signals that feed into recommendation and moderation engines. In theory, good behavior (e.g., honest, accurate, high-quality) begets high reputation, while poor behavior is deterred and pushed off the platform. In this talk, I will discuss how these systems seem to fulfill this mission only coarsely: reputation metrics alone are not strong predictors of success, nor disappearance, nor misbehavior. On one platform, we were able to predict 2 times more suspensions than the reputation system in place using other public signals. In another platform, we found that users with high reputation signals were suspended at significantly lower rates (up to 3 times less) for the same number of offenses and behavior as regular users, which may be counterproductive to platform health. Our results show that crowds develop various heuristics to flag low-quality/malicious, often disregarding reputation and leaving much room for improvement. Speaker(s): Alejandro Cuevas Villalba , Room: 5th floor conference room (Room 538-539), 130 N. Bellefield , Pittsburgh, Pennsylvania, United States, 15260