demonstration

ModelLens: An Interactive System to Support the Model Improvement Practices of Data Science Teams

Authors:

CSCW '19 Companion: Companion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social Computing

Pages 9 - 13

https://doi.org/10.1145/3311957.3359512

Published: 09 November 2019 Publication History

Get Access

Abstract

This demo presents ModelLens, an interactive system designed to support data science teams in their model improvement practices. A central component of improving models is analyzing model errors, often incorporating feedback on the model's precision and recall performance (e.g., differences between the model's predicted label and its actual label). Today, error analysis is typically an ad hoc, team-specific process, largely accomplished through spreadsheets. ModelLens offers a unified view of feedback from multiple sources, the ability for data scientists to explore the context of an individual feedback instance, as well as a customizable ontology to enable collaborative and systematic annotations of model errors.

References

[1]

Kjeld Schmidt and Carla Simonee. 1996. Coordination mechanisms: Towards a conceptual foundation of CSCW systems design. Computer Supported Cooperative Work (CSCW) 5, 2 (1996), 155--200.

Digital Library

Google Scholar

[2]

Marc Steen. 2013. Co-Design as a Process of Joint Inquiry and Imagination. Design Issues 29, 2 (2013), 16--28.

Crossref

Google Scholar

Cited By

View all

Muralikumar MMcDonald D(2024)Analyzing Collaborative Challenges and Needs of UX Practitioners when Designing with AI/MLProceedings of the ACM on Human-Computer Interaction10.1145/36869868:CSCW2(1-25)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686986
Lim GKim HChoi YLi TKulkarni CSubramonyam HSeering JBernstein MZhang AGlassman EPerrault SKim J(2023)Designing for AI-Powered Social Computing SystemsCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3606951(572-575)Online publication date: 14-Oct-2023
https://dl.acm.org/doi/10.1145/3584931.3606951
Heger AMarquis LVorvoreanu MWallach HWortman Vaughan J(2022)Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and DesiderataProceedings of the ACM on Human-Computer Interaction10.1145/35557606:CSCW2(1-29)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555760
Show More Cited By

Index Terms

ModelLens: An Interactive System to Support the Model Improvement Practices of Data Science Teams

Recommendations

iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models
IUI '22: Proceedings of the 27th International Conference on Intelligent User Interfaces

Error analysis in NLP models is essential to successful model development and deployment. One common approach for diagnosing errors is to identify subpopulations in the dataset where the model produces the most errors. However, existing approaches ...
A web-based multi-user error correction, analysis, and feedback system
DIWEB'07: Proceedings of the 7th WSEAS international conference on Distance learning and web engineering

Taking C programming language learning as an example, this paper describes the development of a Web-based multi-user error correction, analysis, and feedback system. This system reinvented traditional paper-based error feedback and error correction in ...
Collaborative Practices with Structured Data: Do Tools Support What Users Need?
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Collaborative work with data is increasingly common and spans a broad range of activities - from creating or analysing data in a team, to sharing it with others, to reusing someone else's data in a new context. In this paper, we explore collaboration ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CSCW '19 Companion: Companion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social Computing

November 2019

562 pages

ISBN:9781450366922

DOI:10.1145/3311957

General Chairs:
Eric Gilbert
University of Michigan
,
Karrie Karahalios
University of Illinois, Urbana- Champaign

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2019

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

CSCW '19

Sponsor:

SIGCHI

CSCW '19: Computer Supported Cooperative Work and Social Computing

November 9 - 13, 2019

TX, Austin, USA

Acceptance Rates

CSCW '19 Companion Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
320
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)1

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Muralikumar MMcDonald D(2024)Analyzing Collaborative Challenges and Needs of UX Practitioners when Designing with AI/MLProceedings of the ACM on Human-Computer Interaction10.1145/36869868:CSCW2(1-25)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686986
Lim GKim HChoi YLi TKulkarni CSubramonyam HSeering JBernstein MZhang AGlassman EPerrault SKim J(2023)Designing for AI-Powered Social Computing SystemsCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing10.1145/3584931.3606951(572-575)Online publication date: 14-Oct-2023
https://dl.acm.org/doi/10.1145/3584931.3606951
Heger AMarquis LVorvoreanu MWallach HWortman Vaughan J(2022)Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and DesiderataProceedings of the ACM on Human-Computer Interaction10.1145/35557606:CSCW2(1-29)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555760
Piorkowski DPark SWang AWang DMuller MPortnoy F(2021)How AI Developers Overcome Communication Challenges in a Multidisciplinary TeamProceedings of the ACM on Human-Computer Interaction10.1145/34492055:CSCW1(1-25)Online publication date: 22-Apr-2021
https://dl.acm.org/doi/10.1145/3449205

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models

A web-based multi-user error correction, analysis, and feedback system

Collaborative Practices with Structured Data: Do Tools Support What Users Need?