More Web Proxy on the site http://driver.im/

Article

The user in experimental computer systems research

Authors:

Peter A. Dinda,

Robert P. Dick,

Arindam Mallik,

Samuel RossoffAuthors Info & Claims

ExpCS '07: Proceedings of the 2007 workshop on Experimental computer science

Pages 10 - es

https://doi.org/10.1145/1281700.1281710

Published: 13 June 2007 Publication History

Abstract

Experimental computer systems research typically ignores the end-user, modeling him, if at all, in overly simple ways. We argue that this (1) results in inadequate performance evaluation of the systems, and (2) ignores opportunities. We summarize our experiences with (a) directly evaluating user satisfaction and (b) incorporating user feedback in different areas of client/server computing, and use our experiences to motivate principles for that domain. Specifically, we report on user studies to measure user satisfaction with resource borrowing and with different clock frequencies in desktop computing, the development and evaluation of user interfaces to integrate user feedback into scheduling and clock frequency decisions in this context, and results in predicting user action and system response in a remote display system. We also present initial results on extending our work to user control of scheduling and mapping of virtual machines in a virtualization-based distributed computing environment. We then generalize (a) and (b) as recommendations for incorporating the user into experimental computer systems research.

References

[1]

Anderson, T. E., Culler, D. E., and Patterson, D. A. A case for networks of workstations. IEEE Micro (February 1995).

Digital Library

[2]

Barratto, R., Kim, L., and Nieh, J. Thine: A virtual display architecture for thin-client computing. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP) (October 2005).

Digital Library

[3]

Bhola, S., and Ahamad, M. Workload modeling for highly interactive applications. In ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (1999), pp. 210--211. Extended version as Technical Report GIT-CC-99-2, College of Computing, Georgia Tech.

Digital Library

[4]

Brakmo, L. S., O'Malley, S. W., and Peterson, L. L. TCP Vegas: New Techniques for Congestion Detection and Avoidance. In Proceedings of the Conference on Communications Architectures, Protocols and Applications (1994), pp. 24--35.

Digital Library

[5]

Card, S., Moran, T., and Newell, A. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Publishers, 1986.

Digital Library

[6]

Chandra, R., Zeldovich, N., Sapuntzakis, C., and Lam, M. The collective: A cache-based system management architecture. In Proceedings of the 2nd Symposium on Networked Systems Design and Implementation (NSDI) (May 2005).

Digital Library

[7]

Chapman, M. rdesktop: A remote desktop protocol client for accessing windows nt terminal server. http://www.rdesktop.org.

[8]

Chien, A. A., Calder, B., Elbert, S., and Bhatia, K. Entropia: architecture and performance of an enterprise desktop grid system. Journal of Parallel and Distributed Computing 63, 5 (2003), 597--610.

Digital Library

[9]

Curtin, M., and Dolske, J. A brute force search of DES keyspace. ;login: (May 1998).

[10]

Dinda, P. A. The statistical properties of host load. Scientific Programming 7, 3, 4 (1999). A version of this paper is also available as CMU Technical Report CMU-CS-TR-98-175. A much earlier version appears in LCR '98 and as CMU-CS-TR-98-143.

Digital Library

[11]

Dourish, P. Evolution in the adoption and use of collaborative technologies.

[12]

Embley, D. W., and Nagy, G. Behavioral aspects of text editors. ACM Computing Surveys 13, 1 (January 1981), 33--70.

Digital Library

[13]

Endo, Y., Wang, Z., Chen, J. B., and Seltzer, M. Using latency to evaluate interactive system performance. In Proceedings of the 1996 Symposium on Operating Systems Design and Implementation (1996).

Digital Library

[14]

Fall, K., and Floyd, S. Simulation-based comparisons of Tahoe, Reno and SACK TCP. SIGCOMM Computer Communication Review 26, 3 (1996), 5--21.

Digital Library

[15]

Flautner, K., and Mudge, T. Vertigo: Automatic Performance-setting for Linux. SIGOPS Oper. Syst. Rev. 36, SI (2002), 105--116. http://doi.acm.org/10.1145/844128.844139.

Digital Library

[16]

Frey, J., Tannenbaum, T., Foster, I., Livny, M., and Tuecke, S. Condor-g: A computation management agent for multi-institutional grids. In Proceedings of the 10th International Symposium on High Performance Distributed Computing (HPDC 2001) (2001), pp. 55--66.

Digital Library

[17]

Gerbessiotis, A. V., and Valiant, L. G. Direct bulk-synchronous parallel algorithms. Journal of Parallel and Distributed Computing 22, 2 (1994), 251--267.

Digital Library

[18]

Google Corporation. Google compute. http://toolbar.google.com/dc/.

[19]

Gupta, A., Lin, B., and Dinda, P. A framework and toolkit for understanding user comfort with resource borrowing. Tech. Rep. NWU-CS-04-28, Department of Computer Science, Northwestern University, February 2004.

[20]

Gupta, A., Lin, B., and Dinda, P. A. Measuring and understanding user comfort with resource borrowing. In Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing (HPDC 2004) (June 2004).

Digital Library

[21]

Huber, P. Robust Statistics. Wiley and Sons, 2003.

[22]

Klein, J. T. Computer response to user frustration. Master's thesis, Massachusetts Institute of Technology, 1999.

[23]

Komatsubara, A. Psychological upper and lower limits of system response time and user's preferance on skill level. In Proceedings of the 7th International Conference on Human Computer Interaction (HCI International 97) (August 1997), G. Salvendy, M. J. Smith, and R. J. Koubek, Eds., vol. 1, IEE, pp. 829--832.

Digital Library

[24]

Kuniavsky, M. Observing the User Experience: A Practioner's Guide to User Research. Morgan Kaufmann, 2003.

Digital Library

[25]

Lai, A., and Nieh, J. Limits of wide-area thin-client computing. In Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (2002).

Digital Library

[26]

Larson, S. M., Snow, C. D., Shirts, M., and Pande, V. S. Folding@home and genome@home: Using distributed computing to tackle previously intractable problems in computational biology. In Computational Genomics, R. Grant, Ed. Horizon Press, 2002.

[27]

Lin, B. Human-directed adaptation. Thesis Proposal, Department of Electrical Engineering and Computer Science, Northwestern University, October 2005.

[28]

Lin, B., and Dinda, P. User-driven scheduling of interactive virtual machines. In Proceedings of the Fifth International Workshop on Grid Computing (November 2004).

Digital Library

[29]

Lin, B., and Dinda, P. Vsched: Mixing batch and interactive virtual machines using periodic real-time scheduling. In Proceedings of ACM/IEEE SC (Supercomputing) (November 2005).

Digital Library

[30]

Lin, B., and Dinda, P. Putting the user in direct control of cpu scheduling. Tech. Rep. NWU-EECS-06-07, Department of Electrical Engineering and Computer Science, Northwestern University, July 2006.

[31]

Lin, B., and Dinda, P. Towards scheduling virtual machines based on direct user input. In Proceedings of the 1st International Workshop on Virtualization Technology in Distributed Computing (VTDC) (2006).

Digital Library

[32]

Lin, B., Mallik, A., Dinda, P., Memik, G., and Dick, R. Power reduction through measurement and modeling of users and cpus: Summary. In Proceedings of the 2007 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (June 2007).

Digital Library

[33]

Litzkow, M., Livny, M., and Mutka, M. W. Condor -- a hunter of idle workstations. In Proceedings of the 8th International Conference of Distributed Computing Systems (ICDCS '88) (June 1988), pp. 104--111.

[34]

Liu, C. L., and Layland, J. W. Scheduling algorithms for multiprogramming in a hard real-time environment. Journal of the ACM 20, 1 (January 1973), 46--61.

Digital Library

[35]

Liu, J. Real-time Systems. Prentice Hall, 2000.

[36]

Lorch, J. R., and Smith, A. J. Using User Interface Event Information in Dynamic Voltage Scaling Algorithms. In Technical Report UCB/CSD-02-1190, Computer Science Division, EECS, University of California at Berkeley, August (2002). citeseer. ist.psu.edu/lorch03using.html.

Digital Library

[37]

Maclean, A., Carter, K., Lovstrand, L., and Moran, T. User-tailorable systems: pressing the issues with buttons. In CHI '90: Proceedings of the SIGCHI conference on Human factors in computing systems (New York, NY, USA, 1990), ACM Press, pp. 175--182.

Digital Library

[38]

Mallik, A., Lin, B., Dinda, P., Memik, G., and Dick, R. Process and user driven dynamic voltage and frequency scaling. Tech. Rep. NWU-EECS-06-11, Department of Electrical Engineering and Computer Science, Northwestern University, August 2006.

[39]

Mallik, A., Lin, B., Memik, G., Dinda, P., and Dick, R. User-driven frequency scaling. IEEE Computer Architecture Letters 5, 2 (2006).

Digital Library

[40]

Microsoft. Remote desktop protocol (rdp) features and performance. Tech. rep., 2000.

[41]

Mutka, M. W., and Livny, M. The available capacity of a privately owned workstation environment. Performance Evaluation 12, 4 (July 1991), 269--284.

Digital Library

[42]

Proctor, R., and Van Zandt, T. Human Factors in Simple and Complex Systems. Allyn and Bacon, 1993.

[43]

Reynolds, C. J. The sensing and measurement of frustration with computers. Master's thesis, Massachusetts Institute of Technology Media Laboratory, 2001. http://www.media.mit.edu/~carsonr/pdf/sm_thesis.pdf.

[44]

Richardson, T., Stafford-Fraser, Q., Wood, K., and Hopper, A. Virtual network computing. IEEE Internet Computing 2, 1 (January/February 1998).

Digital Library

[45]

Romano, P. Itu-t recommendation t.128 (application sharing). Tech. rep., ITU, March 1997.

[46]

Rossoff, S., and Dinda, P. Prospects for speculative remote display. Tech. Rep. NWU-EECS-06-08, Department of Electrical Engineering and Computer Science, Northwestern University, August 2006.

[47]

Ruth, P., McGachey, P., Jiang, X., and Xu, D. Viocluster: Virtualization for dynamic computational domains. In Proceedings of the IEEE International Conference on Cluster Computing (Cluster) (September 2005).

[48]

Satyanarayanan, M., Kozuch, M., Helfrich, C., and O'Hallaron, D. Towards seamless mobility on pervasive hardware. Pervasive and Mobile Computing 1, 2 (June 2005), 157--189.

Digital Library

[49]

Shoykhet, A., Lange, J., and Dinda, P. Virtuoso: A system for virtual machine marketplaces. Tech. Rep. NWU-CS-04-39, Department of Computer Science, Northwestern University, July 2004.

[50]

Sousa, J., Balan, R., Poladian, V., Garlan, D., and Satyanarayanan, M. Giving users the steering wheel for guiding resource-adaptive systems. Tech. Rep. CMU-CS-05-198, Department of Computer Science, Carnegie Mellon University, December 2005.

[51]

Stevens, W. TCP Slow Start, Congestion Avoidance, Fast Retransmit and Fast Recovery Algorithms. In Internet RFC 2001 (1997).

Digital Library

[52]

Stricker, L. J. The true deceiver. Psychological Bulletin, 68 (1967), 13--20.

[53]

Sullivan, W. T., Werthimer, D., Bowyer, S., Cobb, J., Gedye, D., and Anderson, D. A new major seti project based on project serendip data and 100,000 personal computers. In Proceedings of the Fifth International Conference on Bioastronomy (1997), C. Cosmovici, S. Bowyer, and D. Werthimer, Eds., no. 161 in IAU Colloquim, Editrice Compositori, Bologna, Italy.

[54]

Sundararaj, A. Automatic, Run-time, and Dynamic Adaptation of Distributed Applications Executing in Virtual Environments. PhD thesis, Northwestern University, December 2006. Technical Report NWU-EECS-06-18, Department of Electrical Engineering and Computer Science.

Digital Library

[55]

Sundararaj, A., Gupta, A., and Dinda, P. Increasing application performance in virtual environments through run-time inference and adaptation. In Proceedings of the 14th IEEE International Symposium on High Performance Distributed Computing (HPDC) (July 2005).

Digital Library

[56]

Sundararaj, A., Sanghi, M., Lange, J., and Dinda, P. An optimization problem in adaptive virtual environments. In Proceedings of the Seventh Workshop on Mathematical Performance Modeling and Analysis (MAMA) (June 2005).

Digital Library

[57]

Sundararaj, A., Sanghi, M., Lange, J., and Dinda, P. Hardness of approximation and greedy algorithms for the adaptation problem in virtual environments. In Proceedings of the 3rd IEEE International Conference on Autonomic Computing (ICAC) (2006).

Digital Library

[58]

Theocharous, G., Mannor, S., Shah, N., Gandhi, P., Kveton, B., Siddiqi, S., and Yu, C.-H. Machine learning for adaptive power management. Intel Technology Journal 10, 4 (Nov. 2006)

[59]

Wang, Z., and Crowcroft, J. Eliminating Periodic Packet Losses in the 4.3-Tahoe BSD TCP Congestion Control Algorithm. In ACM Computer Communications Review (1992).

Digital Library

[60]

Yan, L., Zhong, L., and Jha, N. K. User-perceived Latency based Dynamic Voltage Scaling for Interactive Applications. In Proceedings of ACM/IEEE Design Automation Conference (2005).

Digital Library

Cited By

Kosch TWelsch RChuang LSchmidt A(2023)The Placebo Effect of Artificial Intelligence in Human–Computer InteractionACM Transactions on Computer-Human Interaction10.1145/352922529:6(1-32)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3529225
Andres JSemertzidis NLi ZWang YFloyd Mueller F(2023)Integrated Exertion—Understanding the Design of Human–Computer Integration in an Exertion ContextACM Transactions on Computer-Human Interaction10.1145/352835229:6(1-28)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3528352
Lachand-Pascal VMichel CSerna ATabard A(2023)Challenges and Opportunities for Multi-Device Management in ClassroomsACM Transactions on Computer-Human Interaction10.1145/351902529:6(1-27)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3519025
Show More Cited By

Index Terms

The user in experimental computer systems research

Recommendations

The user in experimental computer systems research
ecs'07: Experimental computer science on Experimental computer science

Experimental computer systems research typically ignores the end-user, modeling him, if at all, in overly simple ways. We argue that this (1) results in inadequate performance evaluation of the systems, and (2) ignores opportunities. We summarize our ...
Data Pattern for Allocating User Experience Meta-Data to User Experience Research Data
Proceedings of the Symposium on Human Interface 2009 on ConferenceUniversal Access in Human-Computer Interaction. Part I: Held as Part of HCI International 2009

The vision of user experience is making life of users of products as convenient as possible, especially during the interaction with a productor a service. An important aspect of perceived convenience is the user experience of a product. The visual ...
Exploring individual user satisfaction within user-led development

User-led development is gaining popularity with organizations wishing to increase user involvement and control. Typically in this approach a small group of users is given the responsibility for managing the project and representing the user community in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ExpCS '07: Proceedings of the 2007 workshop on Experimental computer science

June 2007

218 pages

ISBN:9781595937513

DOI:10.1145/1281700

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ExpCS07

Sponsor:

ExpCS07: Workshop on Experimental Computer Science

June 13 - 14, 2007

California, San Diego

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
422
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kosch TWelsch RChuang LSchmidt A(2023)The Placebo Effect of Artificial Intelligence in Human–Computer InteractionACM Transactions on Computer-Human Interaction10.1145/352922529:6(1-32)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3529225
Andres JSemertzidis NLi ZWang YFloyd Mueller F(2023)Integrated Exertion—Understanding the Design of Human–Computer Integration in an Exertion ContextACM Transactions on Computer-Human Interaction10.1145/352835229:6(1-28)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3528352
Lachand-Pascal VMichel CSerna ATabard A(2023)Challenges and Opportunities for Multi-Device Management in ClassroomsACM Transactions on Computer-Human Interaction10.1145/351902529:6(1-27)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3519025
Salminen JSantos JJung SJansen B(2023)How does an imaginary persona's attractiveness affect designers' perceptions and IT solutions? An experimental study on users' remote working needsInformation Technology & People10.1108/ITP-09-2022-072936:8(196-225)Online publication date: 26-Jul-2023
https://doi.org/10.1108/ITP-09-2022-0729
Salminen JNielsen LBahloul MJørgensen RSantos JJung SJansen B(2022)Persona preparedness: a survey instrument for measuring the organizational readiness for deploying personasInformation Technology and Management10.1007/s10799-022-00373-925:2(173-198)Online publication date: 13-Sep-2022
https://doi.org/10.1007/s10799-022-00373-9
Feitelson D(2015)From Repeatability to Reproducibility and CorroborationACM SIGOPS Operating Systems Review10.1145/2723872.272387549:1(3-11)Online publication date: 20-Jan-2015
https://dl.acm.org/doi/10.1145/2723872.2723875
Zakay NFeitelson D(2015)Semi-Open Trace Based Simulation for Reliable Evaluation of Job Throughput and User ProductivityProceedings of the 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom)10.1109/CloudCom.2015.35(413-421)Online publication date: 30-Nov-2015
https://dl.acm.org/doi/10.1109/CloudCom.2015.35
Hanna NRichards DHitchens MJacobson M(2014)Towards Quantifying Player's Involvement in 3D Games Based-on Player TypesProceedings of the 2014 Conference on Interactive Entertainment10.1145/2677758.2677763(1-10)Online publication date: 2-Dec-2014
https://dl.acm.org/doi/10.1145/2677758.2677763
Lange JLabrinidis AChrysanthis P(2014)Towards automated personalized data storage2014 IEEE 30th International Conference on Data Engineering Workshops10.1109/ICDEW.2014.6818341(278-283)Online publication date: Mar-2014
https://doi.org/10.1109/ICDEW.2014.6818341
Bongo L(2013)Bulk synchronous visualizationProceedings of the 2013 International Workshop on Programming Models and Applications for Multicores and Manycores10.1145/2442992.2442995(21-30)Online publication date: 23-Feb-2013
https://dl.acm.org/doi/10.1145/2442992.2442995
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents