More Web Proxy on the site http://driver.im/

research-article

Does context matter in quality evaluation of mobile television?

Authors:

Satu Jumisko-Pyykkö,

Miska M. HannukselaAuthors Info & Claims

MobileHCI '08: Proceedings of the 10th international conference on Human computer interaction with mobile devices and services

Pages 63 - 72

https://doi.org/10.1145/1409240.1409248

Published: 02 September 2008 Publication History

Abstract

Subjective quality evaluation is used to optimize the produced audiovisual quality from fundamental signal processing algorithms to consumer services. These studies typically follow the basic principles of controlled psychoperceptual experiments. However, when compromising compression and transmission parameters for consumer services, the ecological validity of conventional quality evaluation methods can be questioned. To tackle this, we firstly present a novel user-oriented quality evaluation method for mobile television in its usage contexts. Secondly, we present the results of an experiment conducted with 30 participants comparing acceptability and satisfaction of quality as well as goals of viewing in three mobile contexts and under four different residual transmission error rates, when the participants also performed simultaneous assessment tasks. Finally, we compare the results with a previous laboratory experiment. The studied error rates impacted negatively on all measured tasks with some contextual differences. Moreover, the evaluations were more favorable and less discriminate in the mobile contexts compared to the laboratory.

References

[1]

Carlsson, C., Walden, P. Mobile TV - To Live or Die by Content, Proc 40th HICSS (2007) 51b.

Digital Library

[2]

Casey, B., Casey, N., Calvert, B., French, L., Justin, L. Television Studies -- The Key Concepts. Routledge. London, 2002.

[3]

Chen, S. Y., Ghinea, G., Macredie, R. D. A Cognitive Approach to User Perception of Multimedia Quality: An Empirical Investigation. International Journal of Human-Computer Studies, 64(12), (2006)1200--1213.

Digital Library

[4]

Coolican, H. Research methods and statistics in psychology, 4th ed, London: J. W. Arrowsmith Ltd, 2004.

[5]

DVB-H Global Mobile TV: Services, Trials & Pilots. http://www.dvb-h.com/services.htm

[6]

ETSI, Digital Video Broadcasting (DVB): DVB specification for data broadcasting, ETSI standard, EN 301 192 V1.4.1, 2004.

[7]

ETSI, Digital Video Broadcasting (DVB): Transmission systems for handheld terminals, ETSI standard, EN 302 304 V1.1.1, 2004.

[8]

ETSI, Digital Video Broadcasting (DVB); Specification for the use of video and audio coding in DVB services delivered directly over IP, ETSI standard, ETSI TS 102 005 V1.2.0, 2005.

[9]

Finnpanel. www.finnpanel.fi, visited 20.5.2006.

[10]

Ghinea, G & Chen, S. Y. The impact of cognitive styles on perceptual distributed multimedia quality. British Journal of Educational Technology. Vol 34, 4 (2003). 393--406.

[11]

Ghinea, G. & Thomas, J. P. QoS impact user perception and understanding of multimedia video clips, Proc. of ACM Multimedia '98". (1998) 49--54

Digital Library

[12]

Goldstein, E. B. Sensation and Perception. United States of America: Wadsworth, 2002.

[13]

Gulliver, G., Ghinea, G. Stars in Their Eyes, What Eye-Tracking Reveals About Multimedia Perceptual Quality. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans. Vol. 34, 4(2004) 472--482

Digital Library

[14]

Gulliver, S. R., Ghinea, G. Defining User Perception of Distributed Multimedia Quality. ACM Transactions on Multimedia Computing, Communications and Applications, 2(4), (2006) 241--257

Digital Library

[15]

Hands, D. "Multimodal Quality Perception: The Effects of Attending to Content on Subjective Quality Ratings". Proceedings of IEEE 3rd Workshop on Multimedia Signal Processing, 1999 pp. 503--508, Copenhagen, Denmark 1999.

[16]

Hannuksela, M. M., Malamal Vadakital, V. K., Jumisko-Pyykkö, S. Comparison of Error Protection Methods for Audio-Video Broadcast over DVB-H, EURASIP Journal on Advances in Signal Processing, vol. 2007, 2007.

[17]

ISO/IEC 14496--10:2003, Coding of Audiovisual Objects---Part 10: Advanced Video Coding," 2003, also ITU-T Recommendation H.264. Advanced video coding for generic audiovisual services. 2003.

[18]

ISO/IEC 14496-3: Information technology -- Coding of audio-visual objects - Part 3: Audio, including amendment 1: "Bandwidth Extension" and amendment 2: "Parametric Coding for High Quality Audio". 2003.

[19]

ISO 13407 Human-centred design processes for interactive systems. International Standard, the International Organization for Standardization. 1999.

[20]

ITU-R BT.500-11 Methodology for the subjective assessment of the quality of television pictures, International Telecommunications Union -- Radiocommunication sector, 2002.

[21]

ITU-T P.911 Recommendation P.911, Subjective audiovisual quality assessment methods for multimedia application, International Telecommunication Union -- Telecommunication sector, 1998.

[22]

Jumisko, S., Ilvonen, V., Väänänen-Vainio-Mattila, K. The Effect of TV Content in Subjective Assessment of Video Quality on Mobile Devices. Proc. Multimedia on Mobile Devices, IS&T/SPIE Symposium on Electronic Imaging, (2005), 243--254.

[23]

Jumisko-Pyykkö, S. "I would like to see the subtitles and the face or at least hear the voice": Effects of Picture ratio and Audio-video Bitrate Ratio on Perception of Quality in Mobile Television. Personalized and Mobile Digital TV Applications in Springer Multimedia Tools and Applications Series. 2007.

Digital Library

[24]

Jumisko-Pyykkö, S., Häkkinen, J. Evaluation of Subjective Video Quality on Mobile Devices. Proc. ACM Multimedia (2005), 535--538

Digital Library

[25]

Jumisko-Pyykkö, S. Häkkinen, J., Nyman, G. Experienced Quality Factors - Qualitative Evaluation Approach to Audiovisual Quality. Proc IST/SPIE conference Electronic Imaging, Multimedia on Mobile Devices 2007

[26]

Jumisko-Pyykkö, S., Vadakital, V., Liinasuo, M., Hannuksela M. M., 2006. Acceptance of Audiovisual Quality in Erroneous Television Sequences over a DVB-H Channel. Proc. VPQM, USA, January 2006.

[27]

Jumisko-Pyykkö, S., Vinod Kumar, M. V., Korhonen, J. Unacceptability of Instantaneous Errors in Mobile Television: From Annoying Audio to Video. Proc. Mobile HCI 2006, ACM Press (2006), 1--8.

Digital Library

[28]

Kaikkonen, A., Kekäläinen, A., Cankar, M., Kallio, T., Kankainen, A. Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing. Journal of Usability Studies, Vol. 1 (1), (2005)4--17

Digital Library

[29]

Kjeldskov, J., Stage, J., New techniques for usability evaluation of mobile systems, International Journal of Human-Computer Studies, Vol 60, 5--6 (2004) 599--620.

[30]

Knoche, H. O., McCarthy, J. D.: Good news for mobile TV, Wireless world research forum, Proc. WWRF 14, 2005.

[31]

Knoche, H., McCarthy, J. D., Sasse, M. A. Can Small Be Beautiful? Assessing Image Size Requirements for Mobile TV. Proc. ACM Multimedia 2005, (2005), 561--

Digital Library

[32]

Knoche, H., Sasse, M. A. Breaking the news on mobile TV: user requirements of a popular mobile content. Proc IS&T/SPIE Symposium on Electronic Imaging, 2006

[33]

Knoche, H., McCarthy, J., Sasse, M. A. How low can you go? The effect of low resolutions on shot types. Personalized and Mobile Digital TV Applications in Springer Multimedia Tools and Applications Series (2007)

Digital Library

[34]

Lang, A. The limited capacity model of mediated message processing. Journal of Communication, (2000) 50, 46--70.

[35]

McCarthy, J. D., Sasse M. A. and Miras D. Sharp or Smooth?: Comparing the Effect of Quantization vs. Framerate for Streamed Video. Proc. CHI 2004, 535--542.

Digital Library

[36]

O'Hara, K., Mitchell, A. S., and Vorbau, A. Consuming video on mobile devices. Proc. CHI '07. ACM Press (2007), 857--866.

Digital Library

[37]

Oksman, V., Noppari,., Tammela, A., Mäkinen, M., Ollikainen, V., News in mobiles. Comparing text, audio and video. VTT 2007. http://www.vtt.fi/inf/pdf/tiedotteet/2007/T2375.pdf

[38]

Reiter, U. & Jumisko-Pyykköö S. Watch, Press and Catch - Impact of Divided Attention on Requirements of Audiovisual Quality. Proc. 12th HCI Int 2007.

Digital Library

[39]

Repo, P. Hyvonen, K. Pantzar, M. Timonen, P. Inventing Use for a Novel Mobile Service. International Journal of Technology and Human Interaction, vol. 2, 2 (2006) 49--62.

[40]

Rogers E. M. Diffusion of Innovations, 5th ed., New York: Free Press, 2003.

[41]

Södergård C. (ed.). Mobile television -- technology and user experiences, Report on the Mobile --TV Project. Espoo: VTT Publications 506, 2003.

[42]

Tamminen, S., Oulasvirta, A. Toiskallio, K., Kankainen, A. Understanding mobile contexts, Personal and Ubiquitous Computing, vol 8, 2 (2004) 135--143.

[43]

Watson, A., Sasse, M. A. Measuring Perceived Quality of Speech and Video in Multimedia Conferencing Applications. Proc. ACM multimedia 1998. 55--60.

Digital Library

[44]

Winkler, S., Faller, C. Perceived audiovisual quality of low-bitrate multimedia content. IEEE Transactions on Multimedia, vol. 8, no. 5, (2006) 973--980.

Digital Library

[45]

Wynekoop, J. L., Russo, N., L., Studying system development methodologies: an examination of research methods. Information Systems Journal 7, 1(1997)47--65.

Cited By

Ak AGera ANoyes DTmar HKatsavounidis ILe Callet P(2024)Comparison of Crowdsourcing And Laboratory Settings for Subjective Assessment of Video Quality and Acceptability & Annoyance2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10647945(1159-1164)Online publication date: 27-Oct-2024
https://doi.org/10.1109/ICIP51287.2024.10647945
Ak APerrin ANoyes DKatsavounidis ILe Callet P(2023)Video Consumption in Context: Influence of Data Plan Consumption on QoEProceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596474(320-324)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596474
Zhu JAk ADormeval CLe Callet PRahul KSethuraman S(2023)Subjective Test Environments: A Multifaceted Examination of Their Impact on Test ResultsProceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596470(298-302)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596470
Show More Cited By

Index Terms

Does context matter in quality evaluation of mobile television?
1. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

A Hybrid Method for Quality Evaluation in the Context of Use for Mobile (3D) Television

Controlled psychoperceptual quality evaluation experiments are used to assess the excellence of produced audiovisual quality from fundamental signal processing algorithms to consumer services. When compromising produced quality for consumer services, ...
Towards context information refinement for proximity mobile service using quality of context
Mobility '09: Proceedings of the 6th International Conference on Mobile Technology, Application & Systems

Recent technology advances in mobile devices provide abundant opportunities for developers to provide ubiquitous computing services. This can be realized by context information available in the user's environment. However, current models lack formal ...
Content and context for mobile television: Integrating trial, expert and user findings

Mobile television brings together two contemporary social developments: enhanced end-user mobility and new forms of access to media content. DVB-H trial activities worldwide reveal the industry's belief in broadcast mobile television as 'the next big ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MobileHCI '08: Proceedings of the 10th international conference on Human computer interaction with mobile devices and services

September 2008

568 pages

ISBN:9781595939524

DOI:10.1145/1409240

Conference Chairs:
Henri ter Hofte
Telematica Instituut
,
Ingrid Mulder
Telematica Instituut & Rotterdam University

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 September 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MobileHCI08

MobileHCI08: 10th International Conference on Human Computer Interaction with Mobile Devices and Services

September 2 - 5, 2008

Amsterdam, The Netherlands

Acceptance Rates

Overall Acceptance Rate 202 of 906 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
569
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ak AGera ANoyes DTmar HKatsavounidis ILe Callet P(2024)Comparison of Crowdsourcing And Laboratory Settings for Subjective Assessment of Video Quality and Acceptability & Annoyance2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10647945(1159-1164)Online publication date: 27-Oct-2024
https://doi.org/10.1109/ICIP51287.2024.10647945
Ak APerrin ANoyes DKatsavounidis ILe Callet P(2023)Video Consumption in Context: Influence of Data Plan Consumption on QoEProceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596474(320-324)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596474
Zhu JAk ADormeval CLe Callet PRahul KSethuraman S(2023)Subjective Test Environments: A Multifaceted Examination of Their Impact on Test ResultsProceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596470(298-302)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596470
Juszka DPapir ZLi JGao XLe Callet PLi ZLu WYang JWang J(2022)Impact of Content on Subjective Quality of Experience Assessment for 3D VideoProceedings of the 2nd Workshop on Quality of Experience in Visual Multimedia Applications10.1145/3552469.3555717(29-38)Online publication date: 14-Oct-2022
https://dl.acm.org/doi/10.1145/3552469.3555717
Lebreton PYamagishi K(2021)Study on user quitting in the Puffer live TV video streaming service2021 13th International Conference on Quality of Multimedia Experience (QoMEX)10.1109/QoMEX51781.2021.9465394(19-24)Online publication date: 14-Jun-2021
https://doi.org/10.1109/QoMEX51781.2021.9465394
Lebreton PYamagishi K(2020)Study on viewing completion ratio of video streaming2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP)10.1109/MMSP48831.2020.9287091(1-6)Online publication date: 21-Sep-2020
https://doi.org/10.1109/MMSP48831.2020.9287091
Borchert KSeufert AGamboa EHirth MHoßfeld T(2020)In vitro vs in vivo: does the study’s interface design influence crowdsourced video QoE?Quality and User Experience10.1007/s41233-020-00041-26:1Online publication date: 2-Nov-2020
https://doi.org/10.1007/s41233-020-00041-2
Li JKrasula LBaveye YLi ZLe Callet P(2019)AccAnn: A New Subjective Assessment Methodology for Measuring Acceptability and Annoyance of Quality of ExperienceIEEE Transactions on Multimedia10.1109/TMM.2019.290372221:10(2589-2602)Online publication date: Oct-2019
https://doi.org/10.1109/TMM.2019.2903722
Krasula LLe Callet P(2018)Emerging science of QoE in multimedia applications: Concepts, experimental guidelines, and validation of modelsAcademic Press Library in Signal Processing, Volume 610.1016/B978-0-12-811889-4.00004-X(163-209)Online publication date: 2018
https://doi.org/10.1016/B978-0-12-811889-4.00004-X
Huang RWei XZhou LLv CMeng HJin J(2018)A survey of data-driven approach on multimedia QoE evaluationFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-018-6342-712:6(1060-1075)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s11704-018-6342-7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents