Designing Against Discrimination in Online Markets
This article provides a conceptual framework for understanding how platforms’ design and policy choices introduce opportunities for users’ biases to affect how they treat one another. Through empirical review of design-oriented interventions used by a range of platforms, and the synthesis of this review into a taxonomy of thematic categories, the authors hope to prompt greater reflection on the stakes of such decisions made by platforms already, guide platforms’ future decisions, and provide a basis for empirical work measuring the impacts of design decisions on discriminatory outcomes. In Part I, the empirical review of platforms is described, and the strategies used to develop the taxonomy are presented. In Part II, the ten thematic categories that emerged from this review are detailed, and how platforms’ design interventions might mediate or exacerbate users’ biased behaviors is discussed. Part III describes the ethical dimensions of platforms’ design choices.
Platforms that connect users to one another have flourished online in domains as diverse as transportation, employment, dating, and housing. When users interact on these platforms, their behavior may be influenced by preexisting biases, including tendencies to discriminate along the lines of race, gender, and other protected characteristics. In aggregate, such user behavior may result in systematic inequities in the treatment of different groups. While there is uncertainty about whether platforms bear legal liability for the discriminatory conduct of their users, platforms necessarily exercise a great deal of control over how users’ encounters are structured—including who is matched with whom for various forms of exchange, what information users have about one another during their interactions, and how indicators of reliability and reputation are made salient, among many other features. Platforms cannot divest themselves of this power; even choices made without explicit regard for discrimination can affect how vulnerable users are to bias. This Article analyzes ten categories of design and policy choices through which platforms may make themselves more or less conducive to discrimination by users. In so doing, it offers a comprehensive account of the complex ways platforms’ design choices might perpetuate, exacerbate, or alleviate discrimination in the contemporary economy.
"Designing Against Discrimination in Online Markets" by K. EC Levy, S. Barocas Berkeley Technology Law Journal, Vol. 32, 2018