More Web Proxy on the site http://driver.im/

research-article

Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets

Authors:

Teanna Barrett,

Amy ZhangAuthors Info & Claims

FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

Pages 1757 - 1771

https://doi.org/10.1145/3593013.3594114

Published: 12 June 2023 Publication History

Abstract

To investigate the well-observed racial disparities in computer vision systems that analyze images of humans, researchers have turned to skin tone as a more objective annotation than race metadata for fairness performance evaluations. However, the current state of skin tone annotation procedures is highly varied. For instance, researchers use a range of untested scales and skin tone categories, have unclear annotation procedures, and provide inadequate analyses of uncertainty. In addition, little attention is paid to the positionality of the humans involved in the annotation process—both designers and annotators alike—and the historical and sociological context of skin tone in the United States. Our work is the first to investigate the skin tone annotation process as a sociotechnical project. We surveyed recent skin tone annotation procedures and conducted annotation experiments to examine how subjective understandings of skin tone are embedded in skin tone annotation procedures. Our systematic literature review revealed the uninterrogated association between skin tone and race and the limited effort to analyze annotator uncertainty in current procedures for skin tone annotation in computer vision evaluation. Our experiments demonstrated that design decisions in the annotation procedure such as the order in which the skin tone scale is presented or additional context in the image (i.e., presence of a face) significantly affected the resulting inter-annotator agreement and individual uncertainty of skin tone annotations. We call for greater reflexivity in the design, analysis, and documentation of procedures for evaluation using skin tone.

References

[1]

Ali Al-Naji and Javaan Chahl. 2017. Simultaneous tracking of cardiorespiratory signals for multiple persons using a machine vision system with noise artifact removal. IEEE journal of translational engineering in health and medicine 5 (2017), 1–10.

[2]

Ali Al-Naji and Javaan Chahl. 2018. Remote optical cardiopulmonary signal extraction with noise artifact removal, multiple subject detection & long-distance. IEEE Access 6 (2018), 11573–11595.

[3]

Mohammed Aledhari, Rehma Razzak, Reza M Parizi, and Gautam Srivastava. 2021. Multimodal machine learning for pedestrian detection. In 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring). IEEE, 1–7.

[4]

Denise Almeida, Konstantin Shmarko, and Elizabeth Lomas. 2022. The ethics of facial recognition technologies, surveillance, and accountability in an age of artificial intelligence: a comparative analysis of US, EU, and UK regulatory frameworks. AI and Ethics 2, 3 (2022), 377–387.

[5]

Janet Anderson, Charles Otto, Brianna Maze, Nathan Kalka, and James A Duncan. 2019. Understanding confounding factors in face detection and recognition. In 2019 International Conference on Biometrics (ICB). IEEE, 1–8.

[6]

Oneida Arosarena. 2015. Options and challenges for facial rejuvenation in patients with higher fitzpatrick skin phototypes. JAMA Facial Plastic Surgery 17, 5 (2015), 358–359.

[7]

Mariam Attia and Julian Edge. 2017. Be (com) ing a reflexive researcher: a developmental approach to research methodology. Open Review of Educational Research 4, 1 (2017), 33–45.

[8]

Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019. Differential privacy has disparate impact on model accuracy. Advances in neural information processing systems 32 (2019).

[9]

Keivan Bahmani, Richard Plesh, Chinmay Sahu, Mahesh Banavar, and Stephanie Schuckers. 2021. SREDS: A dichromatic separation based measure of skin color. In 2021 IEEE International Workshop on Biometrics and Forensics (IWBF). IEEE, 1–6.

[10]

Josh Beal, Hao-Yu Wu, Dong Huk Park, Andrew Zhai, and Dmitry Kislyuk. 2022. Billion-scale pretraining with vision transformers for multi-task visual representations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 564–573.

[11]

Abeba Birhane. 2021. Algorithmic injustice: a relational ethics approach. Patterns 2, 2 (2021), 100205.

[12]

Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.

[13]

United States Census Bureau. 2020. 2020 Census Results. https://www.census.gov/programs-surveys/decennial-census/decade/2020/2020-census-results.html.

[14]

L Elisa Celis and Vijay Keswani. 2020. Implicit diversity in image summarization. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–28.

Digital Library

[15]

Cheng-Chun Chang, Shi-Tien Hsing, Yung-Chi Chuang, Chien-Ta Wu, Tung-Jing Fang, Kuan-Fu Chen, and Bill Choi. 2018. Robust skin type classification using convolutional neural networks. In 2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA). IEEE, 2011–2014.

[16]

Alain Chardon, Isabelle Cretois, and Colette Hourseau. 1991. Skin colour typology and suntanning pathways. International journal of cosmetic science 13, 4 (1991), 191–208.

[17]

Quan Ze Chen, Daniel S Weld, and Amy X Zhang. 2021. Goldilocks: Consistent crowdsourced scalar annotations with relative uncertainty. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–25.

Digital Library

[18]

Cynthia M Cook, John J Howard, Yevgeniy B Sirotin, Jerry L Tipton, and Arun R Vemury. 2019. Demographic effects in facial recognition and their dependence on image acquisition: An evaluation of eleven commercial systems. IEEE Transactions on Biometrics, Behavior, and Identity Science 1, 1 (2019), 32–41.

[19]

Sasha Costanza-Chock, Inioluwa Deborah Raji, and Joy Buolamwini. 2022. Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 1571–1583.

Digital Library

[20]

Roxana Daneshjou, Kailas Vodrahalli, Roberto A Novoa, Melissa Jenkins, Weixin Liang, Veronica Rotemberg, Justin Ko, Susan M Swetter, Elizabeth E Bailey, Olivier Gevaert, 2022. Disparities in dermatology AI performance on a diverse, curated clinical image set. Science advances 8, 31 (2022), eabq6147.

[21]

S Del Bino, J Sok, E Bessac, and F Bernerd. 2006. Relationship between skin response to ultraviolet exposure and skin color type. Pigment cell research 19, 6 (2006), 606–614.

[22]

Emily Denton, Mark Díaz, Ian Kivlichan, Vinodkumar Prabhakaran, and Rachel Rosen. 2021. Whose ground truth? accounting for individual and collective identities underlying dataset annotation. arXiv preprint arXiv:2112.04554 (2021).

[23]

Emily Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, Hilary Nicole, and Morgan Klaus Scheuerman. 2020. Bringing the people back in: Contesting benchmark machine learning datasets. arXiv preprint arXiv:2007.07399 (2020).

[24]

Djellel Eddine Difallah, Elena Filatova, and Panagiotis G. Ipeirotis. 2018. Demographics and Dynamics of Mechanical Turk Workers. Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (2018).

Digital Library

[25]

Manan Doshi, Jimil Shah, Rahul Soni, and Soni Bhambar. 2022. FHP: Facial and Hair Feature Processor for Hairstyle Recommendation. In 2022 IEEE Fourth International Conference on Advances in Electronics, Computers and Communications (ICAECC). IEEE, 1–4.

[26]

Rafael Falcon, Mauro Patti, Stanislas Brochard-Garnier, G Pacianotto Gouveia, S Torres Acevedo, Thelma Bergot, Rick Alarcon, Corentin Bomstein, Hervé Macudzinski, P Maitre, 2022. Image quality evaluation of video conferencing solutions with realistic laboratory scenes. Electronic Imaging 34, 9 (2022), 318–1.

[27]

Todd Feathers. 2021. Google’s new dermatology app wasn’t designed for people with darker skin. Retrieved August 10 (2021), 2022.

[28]

Thomas B. Fitzpatrick. 1988. The Validity and Practicality of Sun-Reactive Skin Types I Through VI. Archives of Dermatology 124, 6 (06 1988), 869–871. https://doi.org/10.1001/archderm.1988.01670060015008

[29]

Thomas B Fitzpatrick. 1988. The validity and practicality of sun-reactive skin types I through VI. Archives of dermatology 124, 6 (1988), 869–871.

[30]

Manuel B Garcia, Teodoro F Revano, Beau Gray M Habal, Jennifer O Contreras, and John Benedic R Enriquez. 2018. A pornographic image and video filtering application using optimized nudity recognition and detection algorithm. In 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM). IEEE, 1–5.

[31]

Mitchell L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S. Bernstein. 2021. The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 388, 14 pages. https://doi.org/10.1145/3411764.3445423

Digital Library

[32]

Matthew Groh, Caleb Harris, Roxana Daneshjou, Omar Badri, and Arash Koochek. 2022. Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm. arXiv preprint arXiv:2207.02942 (2022).

[33]

Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. 2021. Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1820–1828.

[34]

Yağmur Güçlütürk, Umut Güçlü, Katja Seeliger, Sander Bosch, Rob van Lier, and Marcel A van Gerven. 2017. Reconstructing perceived faces from brain activations with deep adversarial neural decoding. Advances in neural information processing systems 30 (2017).

[35]

Ammar Haider and Nosheen Sabahat. 2022. A Usability and Accuracy Measurement of Smartphones Face Recognition. In 2022 2nd International Conference on Artificial Intelligence (ICAI). IEEE, 19–25.

[36]

Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton Ferrer. 2021. Towards measuring fairness in ai: the casual conversations dataset. IEEE Transactions on Biometrics, Behavior, and Identity Science (2021).

[37]

John Hoberman. 2012. Black and blue: The origins and consequences of medical racism. Univ of California Press.

[38]

John J Howard, Yevgeniy B Sirotin, Jerry L Tipton, and Arun R Vemury. 2021. Reliability and validity of image-based and self-reported skin phenotype metrics. IEEE Transactions on Biometrics, Behavior, and Identity Science 3, 4 (2021), 550–560.

[39]

Eun Seo Jo and Timnit Gebru. 2020. Lessons from archives: Strategies for collecting sociocultural data in machine learning. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 306–316.

Digital Library

[40]

Gabriella Kazai, J. Kamps, and Natasa Milic-Frayling. 2012. The face of quality in crowdsourcing relevance labels: demographics, personality and labeling accuracy. Proceedings of the 21st ACM international conference on Information and knowledge management (2012).

Digital Library

[41]

Zaid Khan and Yun Fu. 2021. One label, one billion faces: Usage and consistency of racial categories in computer vision. In Proceedings of the 2021 acm conference on fairness, accountability, and transparency. 587–597.

Digital Library

[42]

Hannah Kim, Girmaw Abebe Tadesse, Celia Cintas, Skyler Speakman, and Kush Varshney. 2022. Out-of-distribution detection in dermatology using input perturbation and subset scanning. In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE, 1–4.

[43]

Newton M Kinyanjui, Timothy Odonga, Celia Cintas, Noel CF Codella, Rameswar Panda, Prasanna Sattigeri, and Kush R Varshney. 2019. Estimating skin tone and effects on classification performance in dermatology datasets. arXiv preprint arXiv:1910.13268 (2019).

[44]

Brendan F Klare, Mark J Burge, Joshua C Klontz, Richard W Vorder Bruegge, and Anil K Jain. 2012. Face recognition performance: Role of demographic information. IEEE Transactions on information forensics and security 7, 6 (2012), 1789–1801.

Digital Library

[45]

Reeta Koshy, Anisha Gharat, Tejashri Wagh, and Siddesh Sonawane. 2021. A Complexion based Outfit color recommender using Neural Networks. In 2021 International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT). IEEE, 1–7.

[46]

KS Krishnapriya, Gabriella Pangelinan, Michael C King, and Kevin W Bowyer. 2022. Analysis of Manual and Automated Skin Tone Assignments. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 429–438.

[47]

Camila Laranjeira da Silva, João Macedo, Sandra Avila, and Jefersson dos Santos. 2022. Seeing without looking: Analysis pipeline for child sexual abuse datasets. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 2189–2205.

Digital Library

[48]

Minh-Ha Le, Md Sakib Nizam Khan, Georgia Tsaloli, Niklas Carlsson, and Sonja Buchegger. 2020. Anonfaces: Anonymizing faces adjusted to constraints on efficacy and security. In Proceedings of the 19th Workshop on Privacy in the Electronic Society. 87–100.

Digital Library

[49]

JC Lester, JL Jia, L Zhang, GA Okoye, and E Linos. 2020. Absence of images of skin of colour in publications of COVID-19 skin manifestations. British Journal of Dermatology 183, 3 (2020), 593–595.

[50]

Angli Liu, Stephen Soderland, Jonathan Bragg, Christopher H Lin, Xiao Ling, and Daniel S Weld. 2016. Effective crowd annotation for relation extraction. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies. 897–906.

[51]

Xin Liu, Ziheng Jiang, Josh Fromm, Xuhai Xu, Shwetak Patel, and Daniel McDuff. 2021. MetaPhys: few-shot adaptation for non-contact physiological measurement. In Proceedings of the conference on health, inference, and learning. 154–163.

Digital Library

[52]

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision. 3730–3738.

Digital Library

[53]

MWP Maduranga and Dilshan Nandasena. 2022. Mobile-based skin disease diagnosis system using convolutional neural networks (CNN). IJ Image Graphics Signal Process. 3 (2022), 47–57.

[54]

Brianna Maze, Jocelyn Adams, James A Duncan, Nathan Kalka, Tim Miller, Charles Otto, Anil K Jain, W Tyler Niggel, Janet Anderson, Jordan Cheney, 2018. Iarpa janus benchmark-c: Face dataset and protocol. In 2018 international conference on biometrics (ICB). IEEE, 158–165.

[55]

Daniel McDuff, Xin Liu, Javier Hernandez, Erroll Wood, and Tadas Baltrusaitis. 2021. Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 3742–3748.

[56]

Daniel McDuff, Shuang Ma, Yale Song, and Ashish Kapoor. 2019. Characterizing bias in classifiers using generative models. Advances in neural information processing systems 32 (2019).

[57]

Anay Mehrotra and L Elisa Celis. 2021. Mitigating bias in set selection with noisy protected attributes. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 237–248.

Digital Library

[58]

Hanna F Menezes, Arthur SC Ferreira, Eanes T Pereira, and Herman M Gomes. 2021. Bias and Fairness in Face Detection. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, 247–254.

[59]

Arjun Menon. 2023. Leveraging Facial Recognition Technology in Criminal Identification. Interdisciplinary Innovations and Developments towards Smart and Sustainable Industries https://doi. org/10.13052/rp-978-87-7022-828-2 (2023).

[60]

Michele Merler, Nalini Ratha, Rogerio S Feris, and John R Smith. 2019. Diversity in faces. arXiv preprint arXiv:1901.10436 (2019).

[61]

Shiksha Mishra, Puspita Majumdar, Muskan Dosi, Mayank Vatsa, and Richa Singh. 2021. Dual sensor indian masked face dataset. In 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021). IEEE, 1–8.

Digital Library

[62]

David A Molina, Leonardo Causa, and Juan Tapia. 2020. Reduction of bias for gender and ethnicity from face images using automated skin tone classification. In 2020 International Conference of the Biometrics Special Interest Group (BIOSIG). IEEE, 1–5.

[63]

Ellis P Monk. 2019. The color of punishment: African Americans, skin tone, and the criminal justice system. Ethnic and Racial Studies 42, 10 (2019), 1593–1612.

[64]

Ellis P Monk Jr. 2021. The unceasing significance of colorism: Skin tone stratification in the United States. Daedalus 150, 2 (2021), 76–90.

[65]

Ellis P Monk Jr, Jerry Kaufman, and Yadira Montoya. 2021. Skin tone and perceived discrimination: Health and aging beyond the binary in NSHAP 2015. The Journals of Gerontology: Series B 76, Supplement_3 (2021), S313–S321.

[66]

Vidya Muthukumar. 2019. Color-theoretic experiments to understand unequal gender classification accuracy from face images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 0–0.

[67]

Cathy O’neil. 2017. Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.

[68]

Chunjong Park, Anas Awadalla, Tadayoshi Kohno, and Shwetak Patel. 2021. Reliable and trustworthy machine learning for health using dataset shift detection. Advances in Neural Information Processing Systems 34 (2021), 3043–3056.

[69]

Jisoo Park, Hyungjoon Kim, Seonmi Ji, and Eenjun Hwang. 2018. An automatic virtual makeup scheme based on personal color analysis. In Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication. 1–7.

Digital Library

[70]

PRH Perera, ESS Soysa, HRS De Silva, ARP Tavarayan, MP Gamage, and KMLP Weerasinghe. 2021. Virtual Makeover and Makeup Recommendation Based on Personal Trait Analysis. In 2021 3rd International Conference on Advancements in Computing (ICAC). IEEE, 288–293.

[71]

Google Research. 2022. Developing the Monk Skin Tone Scale. https://skintone.google/the-scale.

[72]

Joel Ross, Lilly C. Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who are the crowdworkers?: shifting demographics in mechanical turk. CHI ’10 Extended Abstracts on Human Factors in Computing Systems (2010).

Digital Library

[73]

Tate Ryan-Mosley. 2021. The new lawsuit that shows facial recognition is officially a civil rights issue. https://www.technologyreview.com/2021/04/14/1022676/robert-williams-facial-recognition-lawsuit-aclu-detroit-police/.

[74]

Morgan Klaus Scheuerman, Kandrea Wade, Caitlin Lustig, and Jed R Brubaker. 2020. How we’ve taught algorithms to see identity: Constructing race and gender in image databases for facial analysis. Proceedings of the ACM on Human-computer Interaction 4, CSCW1 (2020), 1–35.

Digital Library

[75]

Zaineb Shah, Syed Ayaz Ali Shah, Aamir Shahzad, Ahmad Fayyaz, Shoaib Khaliq, Ali Zahir, and Goh Chuan Meng. 2022. Deep Learning-Based Forearm Subcutaneous Veins Segmentation. IEEE Access 10 (2022), 42814–42820.

[76]

Tomáš Sixta, Julio CS Jacques Junior, Pau Buch-Cardona, Eduard Vazquez, and Sergio Escalera. 2020. Fairface challenge at eccv 2020: Analyzing bias in face recognition. In Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. Springer, 463–481.

[77]

Radim Spetlík, Jan Cech, and Jiri Matas. 2018. Non-contact reflectance photoplethysmography: Progress, limitations, and myths. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 702–709.

Digital Library

[78]

Shea Swauger. 2020. Our bodies encoded: Algorithmic test proctoring in higher education. Critical digital pedagogy (2020).

[79]

Muhammad Uzair Tariq, Arup Kumar Ghosh, Karla Badillo-Urquiola, Abhiditya Jha, Sanjeev Koppal, and Pamela J Wisniewski. 2018. Designing light filters to detect skin using a low-powered sensor. IEEE.

[80]

Yuushi Toyoda, Gale Lucas, and Jonathan Gratch. 2021. Predicting Worker Accuracy from Nonverbal Behaviour: Benefits and Potential for Algorithmic Bias. In Companion Publication of the 2021 International Conference on Multimodal Interaction. 25–30.

Digital Library

[81]

Alex S Vitale. 2021. The end of policing. Verso Books.

[82]

Wenjin Wang, Albertus C Den Brinker, Sander Stuijk, and Gerard De Haan. 2016. Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering 64, 7 (2016), 1479–1491.

[83]

Olivia R Ware, Jessica E Dawson, Michi M Shinohara, and Susan C Taylor. 2020. Racial limitations of Fitzpatrick skin type. Cutis 105, 2 (2020), 77–80.

[84]

Cameron Whitelam, Emma Taborsky, Austin Blanton, Brianna Maze, Jocelyn Adams, Tim Miller, Nathan Kalka, Anil K Jain, James A Duncan, Kristen Allen, 2017. Iarpa janus benchmark-b face dataset. In proceedings of the IEEE conference on computer vision and pattern recognition workshops. 90–98.

[85]

Benjamin Wilson, Judy Hoffman, and Jamie Morgenstern. 2019. Predictive inequity in object detection. arXiv preprint arXiv:1902.11097 (2019).

[86]

Seyma Yucer, Furkan Tektas, Noura Al Moubayed, and Toby P Breckon. 2022. Measuring hidden bias within face recognition via racial phenotypes. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 995–1004.

[87]

Longhao Zhang, Xipeng Pan, Huihua Yang, and Lingqiao Li. 2020. On Open-Set, High-Fidelity and Identity-Specific Face Transformation. IEEE Access 8 (2020), 224643–224653.

[88]

Xiaobiao Zhang, Xiaoyi Feng, and Zhaoqiang Xia. 2019. Analysis of Factors on BVP Signal Extraction Based on Imaging Principle. In Proceedings of the 2019 3rd International Conference on Biometric Engineering and Applications. 48–55.

Digital Library

[89]

Qian Zheng, Ankur Purwar, Heng Zhao, Guang Liang Lim, Ling Li, Debasish Behera, Qian Wang, Min Tan, Rizhao Cai, Jennifer Werner, 2022. Automatic facial skin feature detection for everyone. arXiv preprint arXiv:2203.16056 (2022).

Cited By

Chandhiramowuli STaylor AHeitlinger SWang D(2024)Making Data Work CountProceedings of the ACM on Human-Computer Interaction10.1145/36373678:CSCW1(1-26)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637367
Alkhathlan MCachel KShrestha HHarrison LRundensteiner E(2024)Balancing Act: Evaluating People’s Perceptions of Fair Ranking MetricsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659018(1940-1970)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659018
Scheuerman M(2024)In the Walled Garden: Challenges and Opportunities for Research on the Practices of the AI Tech IndustryProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658918(456-466)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658918
Show More Cited By

Index Terms

Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Human-centered computing

Recommendations

Skin color enhancement based on favorite skin color in HSV color space

Skin color enhancement based on favorite skin color is proposed to make skin color displayed on large screen flat panel TVs agree with human favorite skin color. A robust skin detection method in different intensity is obtained after analyzing the ...
Skin tone enhancement and background change for mobile phones

In this paper, we propose skin tone enhancement and background change algorithms for mobile phones. It is important to obtain natural results in skin tone enhancement and background change since only parts of an image, skin tone and background, are ...
Consensus and subjectivity of skin tone annotation for ML fairness
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems

Understanding different human attributes and how they affect model behavior may become a standard need for all model creation and usage, from traditional computer vision tasks to the newest multimodal generative AI systems. In computer vision ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

June 2023

1929 pages

ISBN:9798400701924

DOI:10.1145/3593013

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2023

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

FAccT '23

FAccT '23: the 2023 ACM Conference on Fairness, Accountability, and Transparency

June 12 - 15, 2023

IL, Chicago, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)150
Downloads (Last 6 weeks)15

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chandhiramowuli STaylor AHeitlinger SWang D(2024)Making Data Work CountProceedings of the ACM on Human-Computer Interaction10.1145/36373678:CSCW1(1-26)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637367
Alkhathlan MCachel KShrestha HHarrison LRundensteiner E(2024)Balancing Act: Evaluating People’s Perceptions of Fair Ranking MetricsProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659018(1940-1970)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3659018
Scheuerman M(2024)In the Walled Garden: Challenges and Opportunities for Research on the Practices of the AI Tech IndustryProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658918(456-466)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658918
Jaime SKern C(2024)Ethnic Classifications in Algorithmic Fairness: Concepts, Measures and Implications in PracticeProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658902(237-253)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658902
Xiong ZFu SZhu YZhu CMa XFan M(2024)“It is hard to remove from my eye”: Design Makeup Residue Visualization System for Chinese Traditional Opera (Xiqu) PerformersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642261(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642261
Yang LKumar RKaur RBabbar AMakhanshahi GSingh AKumar RBhowmik AAlawadi A(2024)Exploring the role of computer vision in product design and development: a comprehensive reviewInternational Journal on Interactive Design and Manufacturing (IJIDeM)10.1007/s12008-024-01765-718:6(3633-3680)Online publication date: 14-Mar-2024
https://doi.org/10.1007/s12008-024-01765-7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents