short-paper

QUINCE: A unified crowdsourcing-based QoE measurement platform

Authors:

Ricky K. P. Mok,

Ginga Kawaguti,

kc claffyAuthors Info & Claims

SIGCOMM Posters and Demos '19: Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos

Pages 60 - 62

https://doi.org/10.1145/3342280.3342307

Published: 19 August 2019 Publication History

Get Access

Abstract

Assessing QoE in situ is a challenging task. Researchers have employed crowdsourcing-based approaches to achieve scale and diversity of subjects, but not without confounding factors. Apart from subjective bias of users, environmental factors introduce variance to QoE measurements. We propose QUINCE, a QoE measurement platform, which uses a gamified approach to enable longitudinal study with repeated and varying measurements in a single platform. We leveraged existing Internet measurement data and infrastructures to integrate three different types of network and QoE measurements to yield a more comprehensive view of subjects. Our preliminarily results show that QUINCE achieves a high level of engagement from subjects and collects data that is useful for correlating network performance and YouTube video streaming QoE.

References

[1]

Amazon. 2018. Mechanical Turk. https://www.mturk.com.

Google Scholar

[2]

Amogh Dhamdhere, David Clark, Alexander Gamero-Garrido, Matthew Luckie, Ricky Mok, Gautam Akiwate, Kabir Gogia, Vaibhav Bajpai, Alex Snoeren, and kc claffy. 2018. Inferring Persistent Interdomain Congestion. In Proc. ACM SIGCOMM.

Digital Library

Google Scholar

[3]

Qingzhu Gao, Prasenjit Dey, and Parvez Ahammad. 2017. Perceived Performance of Top Retail Webpages In the Wild. ACM SIGCOMM Computer Communication Review 47, 5 (Oct 2017), 42--47.

Digital Library

Google Scholar

[4]

Bruno Gardlo, Sebastian Egger, Michael Seufert, and Raimund Schatz. 2014. Crowdsourcing 2.0: Enhancing Execution Speed and Reliability of Web-based QoE Testing. In Proc. IEEE ICC.

Crossref

Google Scholar

[5]

Gokay Huz, Steven Bauer, kc claffy, and Robert Beverly. 2015. Experience in using MTurk for Network Measurement. In Proc. ACM C2B(1)D.

Digital Library

Google Scholar

[6]

Matteo Varvello, Jeremy Blackburn, David Naylor, and Konstantina Papagiannaki. 2016. EYEORG: A Platform For Crowdsourcing Web Quality Of Experience Measurements. In Proc. ACM CoNEXT.

Digital Library

Google Scholar

[7]

YouTube. 2018. YouTube Player API Reference for iframe Embeds. https://developers.google.com/youtube/iframe_api_reference.

Google Scholar

[8]

Lingyan Zhang, Shangguang Wang, Fangchun Yang, and Rong N. Chang. 2017. QoECenter: A Visual Platform for QoE Evaluation of Streaming Video Services. In Proc. IEEE ICWS.

Google Scholar

Cited By

View all

Cieplinska NJanowski LDe Moor KWierzchon M(2022)Long-Term Video QoE Assessment Studies: A Systematic ReviewIEEE Access10.1109/ACCESS.2022.323174710(133883-133897)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3231747
Abdelwahed NLetaifa AAsmi S(2022)Monitoring web QoE based on analysis of client-side measures and user behaviorMultimedia Tools and Applications10.1007/s11042-022-13427-582:4(6243-6269)Online publication date: 5-Aug-2022
https://dl.acm.org/doi/10.1007/s11042-022-13427-5
Huet ASaverimoutou AHouidi ZShi HCai SXu JMathieu BRossi D(2021)Deployable Models for Approximating Web QoE Metrics From Encrypted TrafficIEEE Transactions on Network and Service Management10.1109/TNSM.2021.307367218:3(3336-3352)Online publication date: Sep-2021
https://doi.org/10.1109/TNSM.2021.3073672

Index Terms

QUINCE: A unified crowdsourcing-based QoE measurement platform
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Web-based interaction
2. Networks
  1. Network performance evaluation
    1. Network measurement

Recommendations

Experience in using MTurk for Network Measurement
C2B(1)D '15: Proceedings of the 2015 ACM SIGCOMM Workshop on Crowdsourcing and Crowdsharing of Big (Internet) Data

Conducting sound measurement studies of the global Internet is inherently difficult. The collected data significantly depends on vantage point(s), sampling strategies, security policies, or measurement populations -- and conclusions drawn from the data ...
QDASH: a QoE-aware DASH system
MMSys '12: Proceedings of the 3rd Multimedia Systems Conference

Dynamic Adaptation Streaming over HTTP (DASH) enhances the Quality of Experience (QoE) for users by automatically switching quality levels according to network conditions. Various adaptation schemes have been proposed to select the most suitable quality ...
A QoE management system to improve the IPTV network

One of the biggest problems of the IPTV providers is to offer enough quality of experience (QoE) to their customers. The minimum bandwidth in the access network required to provide IPTV services, jointly with the necessity to guarantee the QoE to the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGCOMM Posters and Demos '19: Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos

August 2019

183 pages

ISBN:9781450368865

DOI:10.1145/3342280

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

SIGCOMM '19

Sponsor:

SIGCOMM

SIGCOMM '19: ACM SIGCOMM 2019 Conference

August 19 - 23, 2019

Beijing, China

Acceptance Rates

SIGCOMM Posters and Demos '19 Paper Acceptance Rate 62 of 102 submissions, 61%;

Overall Acceptance Rate 92 of 158 submissions, 58%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
154
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cieplinska NJanowski LDe Moor KWierzchon M(2022)Long-Term Video QoE Assessment Studies: A Systematic ReviewIEEE Access10.1109/ACCESS.2022.323174710(133883-133897)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3231747
Abdelwahed NLetaifa AAsmi S(2022)Monitoring web QoE based on analysis of client-side measures and user behaviorMultimedia Tools and Applications10.1007/s11042-022-13427-582:4(6243-6269)Online publication date: 5-Aug-2022
https://dl.acm.org/doi/10.1007/s11042-022-13427-5
Huet ASaverimoutou AHouidi ZShi HCai SXu JMathieu BRossi D(2021)Deployable Models for Approximating Web QoE Metrics From Encrypted TrafficIEEE Transactions on Network and Service Management10.1109/TNSM.2021.307367218:3(3336-3352)Online publication date: Sep-2021
https://doi.org/10.1109/TNSM.2021.3073672

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Experience in using MTurk for Network Measurement

QDASH: a QoE-aware DASH system

A QoE management system to improve the IPTV network