research-article

Examining multiple potential models in end-user interactive concept learning

Authors:

Saleema Amershi,

James Fogarty,

Ashish Kapoor,

Desney TanAuthors Info & Claims

CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1357 - 1360

https://doi.org/10.1145/1753326.1753531

Published: 10 April 2010 Publication History

Get Access

Abstract

End-user interactive concept learning is a technique for interacting with large unstructured datasets, requiring insights from both human-computer interaction and machine learning. This note re-examines an assumption implicit in prior interactive machine learning research, that interaction should focus on the question "what class is this object?". We broaden interaction to include examination of multiple potential models while training a machine learning system. We evaluate this approach and find that people naturally adopt revision in the interactive machine learning process and that this improves the quality of their resulting models for difficult concepts.

References

[1]

Amershi, S., Fogarty, J., Kapoor, A., and Tan, D. Overview-Based Example Selection in End-User Interactive Concept Learning. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2009), 247--256.

Digital Library

Google Scholar

[2]

Beygelzimer, A., Dasgupta, S., and Langford, J. Importance Weighted Active Learning. Proceedings of the International Conference on Machine Learning (ICML 2009), 49--56.

Digital Library

Google Scholar

[3]

Dey, A.K., Hamid, R., Beckmann, C., Li, I. and Hsu, D. a CAPpella: Programming by Demonstrations of Context-Aware Applications. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2004), 33--40.

Digital Library

Google Scholar

[4]

Fails, J.A. and Olsen Jr., D.R. Interactive Machine Learning. Proceedings of the ACM Conference on Intelligent User Interfaces (IUI 2003), 39--45.

Digital Library

Google Scholar

[5]

Fogarty, J., Tan, D., Kapoor, A., and Winder, S. CueFlik: Interactive Concept Learning in Image Search. Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2008), 29--38.

Digital Library

Google Scholar

[6]

Kohavi, R. A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 1995), 1137--1143.

Digital Library

Google Scholar

[7]

MacKay, D.J.C. Information-Based Objective Functions for Active Data Selection. Neural Computation 4, 4 (1992), 590--604.

Digital Library

Google Scholar

[8]

Nielsen, J. Heuristic Evaluation. In Nielsen, J. and Mack, R. L. (eds.). Usability Inspection Methods, John Wiley & Sons, New York, NY, 1994.

Digital Library

Google Scholar

[9]

Terry, M. and Mynatt, E.D. Side views: persistent, ondemand previews for open-ended tasks. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2002), 71--80.

Digital Library

Google Scholar

[10]

Tong, S. and Chang, E. Support Vector Machine Active Learning for Image Retrieval. Proceedings of the ACM Conference on Multimedia, (2001), 107--118.

Digital Library

Google Scholar

[11]

Witten, I.H. and Frank, E. Data Mining: Practical Machine Learning Tools and Techniques (2nd Edition), Morgan Kaufmann, San Francisco, CA, USA, 2005.

Digital Library

Google Scholar

Cited By

View all

Chang CTang YYang XChen XIgarashi T(2024)Speed Labeling: Non-stop Scrolling for Fast Image LabelingProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670958(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670958
Zang AXu RTrajcevski GZhou F(2024)Data Issues in High-Definition Maps Furniture – A SurveyACM Transactions on Spatial Algorithms and Systems10.1145/362716010:1(1-37)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3627160
Iglar ASimkute ASellen AChignell M(2024)Getting Back Together: HCI and Human Factors Joining Forces to Meet the AI Interaction ChallengeExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636285(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636285
Show More Cited By

Index Terms

Examining multiple potential models in end-user interactive concept learning
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Overview based example selection in end user interactive concept learning
UIST '09: Proceedings of the 22nd annual ACM symposium on User interface software and technology

Interaction with large unstructured datasets is difficult because existing approaches, such as keyword search, are not always suited to describing concepts corresponding to the distinctions people want to make within datasets. One possible solution is ...
Interactive relational reinforcement learning of concept semantics

We present a framework for the machine learning of denotational concept semantics using a simple form of symbolic interaction of machines with human users. The capability of software agents and robots to learn how to communicate verbally with human ...
Learning models based on formal concept
RSKT'07: Proceedings of the 2nd international conference on Rough sets and knowledge technology

From the classic view in which a concept consists of the set of extents and the set of intents, a concept learning system extended from a formal context is introduced and two concepts such as an under concept and an over concept are defined. Any pair of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2010

2690 pages

ISBN:9781605589299

DOI:10.1145/1753326

General Chair:
Elizabeth Mynatt
Georgia Institute of Technology
,
Program Chairs:
Geraldine Fitzpatrick
Vienna University of Technology
,
Scott Hudson
Carnegie Mellon University
,
Keith Edwards
Georgia Tech
,
Tom Rodden
University of Nottingham

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

end-user interactive concept learning

Qualifiers

Research-article

Conference

CHI '10

Sponsor:

SIGCHI

CHI '10: CHI Conference on Human Factors in Computing Systems

April 10 - 15, 2010

Georgia, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
631
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)2

Reflects downloads up to 30 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chang CTang YYang XChen XIgarashi T(2024)Speed Labeling: Non-stop Scrolling for Fast Image LabelingProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670958(1-10)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670958
Zang AXu RTrajcevski GZhou F(2024)Data Issues in High-Definition Maps Furniture – A SurveyACM Transactions on Spatial Algorithms and Systems10.1145/362716010:1(1-37)Online publication date: 15-Jan-2024
https://dl.acm.org/doi/10.1145/3627160
Iglar ASimkute ASellen AChignell M(2024)Getting Back Together: HCI and Human Factors Joining Forces to Meet the AI Interaction ChallengeExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636285(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636285
Bhattacharya AStumpf SGosak LStiglic GVerbert K(2024)EXMOS: Explanatory Model Steering through Multifaceted Explanations and Data ConfigurationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642106(1-27)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642106
Lai SLuan WTao J(2024)Explore Your Network in Minutes: A Rapid Prototyping Toolkit for Understanding Neural Networks with Visual AnalyticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332657530:1(683-693)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326575
Chang CHe YDu XYang XXie H(2024)Dynamic Labeling: A Control System for Labeling Styles in Image Annotation TasksHuman Interface and the Management of Information10.1007/978-3-031-60107-1_8(99-118)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-60107-1_8
Chignell MWang LZare ALi J(2023)The Evolution of HCI and Human Factors: Integrating Human and Artificial IntelligenceACM Transactions on Computer-Human Interaction10.1145/355789130:2(1-30)Online publication date: 17-Mar-2023
https://dl.acm.org/doi/10.1145/3557891
Dutta PMuppalaneni NPatgiri R(2022)A Survey on Explainability in Artificial IntelligenceHandbook of Research on Advances in Data Analytics and Complex Communication Networks10.4018/978-1-7998-7685-4.ch004(55-75)Online publication date: 2022
https://doi.org/10.4018/978-1-7998-7685-4.ch004
Chang CLee CIgarashi TKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Spatial Labeling: Leveraging Spatial Layout for Improving Label Quality in Non-Expert Image AnnotationProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445165(1-12)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445165
Lee BWeld DIqbal SMacLean KChevalier FMueller S(2020)Newspaper Navigator: Open Faceted Search for 1.5 Million ImagesAdjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379350.3416143(120-122)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3379350.3416143
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Overview based example selection in end user interactive concept learning

Interactive relational reinforcement learning of concept semantics

Learning models based on formal concept