More Web Proxy on the site http://driver.im/

research-article

Testing symmetric properties of distributions

Author:

Paul ValiantAuthors Info & Claims

STOC '08: Proceedings of the fortieth annual ACM symposium on Theory of computing

Pages 383 - 392

https://doi.org/10.1145/1374376.1374432

Published: 17 May 2008 Publication History

Abstract

We introduce the notion of a Canonical Tester for a class of properties on distributions, that is, a tester strong and general enough that "a distribution property in the class is testable if and only if the Canonical Tester tests it". We construct a Canonical Tester for the class of symmetric properties of one or two distributions, satisfying a certain weak continuity condition. Analyzing the performance of the Canonical Tester on specific properties resolves several open problems, establishing lower bounds that match known upper bounds: we show that distinguishing between entropy <α or >β on distributions over [n] requires n^{α/β- o(1)} samples, and distinguishing whether a pair of distributions has statistical distance <α or >β requires n^1-o(1) samples. Our techniques also resolve a conjecture about a property that our Canonical Tester does not apply to: distinguishing identical distributions from those with statistical distance >β requires Ω(n^2/3) samples.

References

[1]

. Alon, A. Andoni, T. Kaufman, K. Matulef, R. Rubinfeld, and N. Xie. Testing k--wise and Almost k--wise Independence. STOC 2007.

Digital Library

[2]

. Alon, E. Fischer, I. Newman, and A. Shapira. A combinatorial characterization of the testable graph properties: it’s all about regularity. STOC 2006.

Digital Library

[3]

. Batu. Testing Properties of Distributions. Ph.D. thesis, Cornell University, 2001.

Digital Library

[4]

. Batu, S. Dasgupta, R. Kumar, and R. Rubinfeld. The complexity of approximating the entropy. STOC, 2002.

Digital Library

[5]

. Batu, E. Fischer, L. Fortnow, R. Kumar, R. Rubinfeld, and P. White. Testing random variables for independence and identity. FOCS, 2001.

Digital Library

[6]

. Batu, L. Fortnow, R. Rubinfeld, W.D. Smith, and P. White. "Testing that distributions are close". FOCS, 2000.

Digital Library

[7]

. Batu, R. Kumar, and R. Rubinfeld. Sublinear algorithms for testing monotone and unimodal distributions. STOC, 2004.

Digital Library

[8]

. Blum and S. Kannan. Designing Programs that Check Their Work. STOC 1989.

Digital Library

[9]

. Blum, M. Luby, and R. Rubinfeld. Self--testing/correcting with applications to numerical problems. Journal of Computer and System Sciences 47(3):549---595, 1993.

Digital Library

[10]

. Chakrabarti, G. Cormode, and A. McGregor. A Near--Optimal Algorithm for Computing the Entropy of a Stream. SODA 2007.

Digital Library

[11]

. Charikar, S. Chaudhuri, R. Motwani, and V.R. Narasayya. Towards Estimation Error Guarantees for Distinct Values. PODS, 2000.

Digital Library

[12]

. Goldreich, S. Goldwasser, and D. Ron. Property Testing and Its Connection to Learning and Approximation. FOCS 1996.

Digital Library

[13]

. Goldreich and L. Trevisan. Three theorems regarding testing graph properties. Random Struct. Algorithms, 23(1):23--57, 2003.

Digital Library

[14]

. Guha, A. McGregor, and S. Venkatasubramanian. Streaming and Sublinear Approximation of Entropy and Information Distances. SODA 2006.

Digital Library

[15]

. Klinger. The Vandermonde Matrix. The American Mathematical Monthly, 74(5):571--574, 1967.

[16]

. Indyk and A. McGregor. Declaring Independence via the Sketching of Sketches. SODA 2008.

Digital Library

[17]

. Indyk and D. Woodruff. Tight Lower Bounds for the Distinct Elements Problem. FOCS, 2003.

Digital Library

[18]

. Raskhodnikova, D. Ron, A. Shpilka, and A. Smith. Strong Lower Bounds for Approximating Distribution Support Size and the Distinct Elements Problem. FOCS 2007.

Digital Library

[19]

. Roos. On the Rate of Multivariate Poisson Convergence. Journal of Multivariate Analysis 69:120--134, 1999.

Digital Library

[20]

. Rubinfeld and M. Sudan. Robust Characterizations of Polynomials with Applications to Program Testing. SIAM Journal on Computing 25(2):252--271, 1996.

Digital Library

[21]

. Valiant. Testing Symmetric Properties of Distributions. ECCC report TR07--135, 2007.

Cited By

Yu N(2023)Almost Tight Sample Complexity Analysis of Quantum Identity Testing by Pauli MeasurementsIEEE Transactions on Information Theory10.1109/TIT.2023.327120669:8(5060-5068)Online publication date: Aug-2023
https://doi.org/10.1109/TIT.2023.3271206
Glisic SLorenzo B(2022)QC OptimizationArtificial Intelligence and Quantum Computing for Advanced Wireless Networks10.1002/9781119790327.ch13(593-635)Online publication date: 15-Apr-2022
https://doi.org/10.1002/9781119790327.ch13
O’Donnell RWright J(2021)Quantum Spectrum TestingCommunications in Mathematical Physics10.1007/s00220-021-04180-1387:1(1-75)Online publication date: 17-Aug-2021
https://doi.org/10.1007/s00220-021-04180-1
Show More Cited By

Index Terms

Testing symmetric properties of distributions
1. Mathematics of computing
  1. Probability and statistics
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

Testing Symmetric Properties of Distributions

We introduce the notion of a canonical tester for a class of properties on distributions, that is, a tester strong and general enough that “a distribution property in the class is testable if and only if the canonical tester tests it.” We construct a ...
Sublinear algorithms for testing monotone and unimodal distributions
STOC '04: Proceedings of the thirty-sixth annual ACM symposium on Theory of computing

The complexity of testing properties of monotone and unimodal distributions, when given access only to samples of the distribution, is investigated. Two kinds of sublinear-time algorithms---those for testing monotonicity and those that take advantage of ...
Testing monotone high-dimensional distributions
STOC '05: Proceedings of the thirty-seventh annual ACM symposium on Theory of computing

A monotone distribution P over a (partially) ordered domain has P(y) ≥ P(x) if y ≥ x in the order. We study several natural problems of testing properties of monotone distributions over the n-dimensional Boolean cube, given access to random draws from ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

STOC '08: Proceedings of the fortieth annual ACM symposium on Theory of computing

May 2008

712 pages

ISBN:9781605580470

DOI:10.1145/1374376

General Chair:
Richard Ladner
University of Washington
,
Program Chair:
Cynthia Dwork
Microsoft Research, Silicon Valley

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 May 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

STOC '08

Sponsor:

STOC '08: Symposium on Theory of Computing

May 17 - 20, 2008

British Columbia, Victoria, Canada

Acceptance Rates

STOC '08 Paper Acceptance Rate 80 of 325 submissions, 25%;

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Upcoming Conference

STOC '25

Sponsor:
sigact

57th Annual ACM Symposium on Theory of Computing (STOC 2025)

June 23 - 27, 2025

Prague , Czech Republic

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
396
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)5

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu N(2023)Almost Tight Sample Complexity Analysis of Quantum Identity Testing by Pauli MeasurementsIEEE Transactions on Information Theory10.1109/TIT.2023.327120669:8(5060-5068)Online publication date: Aug-2023
https://doi.org/10.1109/TIT.2023.3271206
Glisic SLorenzo B(2022)QC OptimizationArtificial Intelligence and Quantum Computing for Advanced Wireless Networks10.1002/9781119790327.ch13(593-635)Online publication date: 15-Apr-2022
https://doi.org/10.1002/9781119790327.ch13
O’Donnell RWright J(2021)Quantum Spectrum TestingCommunications in Mathematical Physics10.1007/s00220-021-04180-1387:1(1-75)Online publication date: 17-Aug-2021
https://doi.org/10.1007/s00220-021-04180-1
Canonne CDiakonikolas IStewart A(2018)Testing for families of distributions via the fourier transformProceedings of the 32nd International Conference on Neural Information Processing Systems10.5555/3327546.3327671(10084-10095)Online publication date: 3-Dec-2018
https://dl.acm.org/doi/10.5555/3327546.3327671
Valiant GValiant P(2017)Estimating the UnseenJournal of the ACM10.1145/312564364:6(1-41)Online publication date: 4-Oct-2017
https://dl.acm.org/doi/10.1145/3125643
Acharya JOrlitsky ASuresh ATyagi H(2017)Estimating Renyi Entropy of Discrete DistributionsIEEE Transactions on Information Theory10.1109/TIT.2016.262043563:1(38-56)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1109/TIT.2016.2620435
Diakonikolas IKane DStewart AWichs DMansour Y(2016)The fourier transform of poisson multinomial distributions and its algorithmic applicationsProceedings of the forty-eighth annual ACM symposium on Theory of Computing10.1145/2897518.2897552(1060-1073)Online publication date: 19-Jun-2016
https://dl.acm.org/doi/10.1145/2897518.2897552
Unnikrishnan JHuang D(2016)Weak Convergence Analysis of Asymptotically Optimal Hypothesis TestsIEEE Transactions on Information Theory10.1109/TIT.2016.256343962:7(4285-4299)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1109/TIT.2016.2563439
Wu YYang P(2016)Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial ApproximationIEEE Transactions on Information Theory10.1109/TIT.2016.254846862:6(3702-3720)Online publication date: Jun-2016
https://doi.org/10.1109/TIT.2016.2548468
Acharya JOrlitsky ASuresh ATyagi HIndyk P(2015)The complexity of estimating Rényi entropyProceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms10.5555/2722129.2722253(1855-1869)Online publication date: 4-Jan-2015
https://dl.acm.org/doi/10.5555/2722129.2722253
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents