More Web Proxy on the site http://driver.im/

Article

Approximating the smallest grammar: Kolmogorov complexity in natural models

Authors:

Moses Charikar,

Rina Panigrahy,

Manoj Prabhakaran,

abhi shelatAuthors Info & Claims

STOC '02: Proceedings of the thiry-fourth annual ACM symposium on Theory of computing

Pages 792 - 801

https://doi.org/10.1145/509907.510021

Published: 19 May 2002 Publication History

Abstract

We consider the problem of finding the smallest context-free grammar that generates exactly one given string of length n. The size of this grammar is of theoretical interest as an efficiently computable variant of Kolmogorov complexity. The problem is of practical importance in areas such as data compression and pattern extraction.The smallest grammar is known to be hard to approximate to within a constant factor, and an o(logn/log logn) approximation would require progress on a long-standing algebraic problem [10]. Previously, the best proved approximation ratio was O(n^1/2) for the Bisection algorithm [8]. Our main result is an exponential improvement of this ratio; we give an O(log (n/g^*)) approximation algorithm, where g^* is the size of the smallest grammar.We then consider other computable variants of Kolomogorov complexity. In particular we give an O(log² n) approximation for the smallest non-deterministic finite automaton with advice that produces a given string. We also apply our techniques to "advice-grammars" and "edit-grammars", two other natural models of string complexity.

References

[1]

A. Apostolico and S. Lonardi. Some theory and practice of greedy off-line textual substitution. In J. A. Storer and M. Cohn, editors, Data Compression Conference, pages 119--128, Snowbird, Utah, 1998.

Digital Library

[2]

V. Chvátal. A greedy heuristic for the set-covering problem. (MATH)ematics of Operations Research, 4(3):233--235, 1979.

[3]

T. M. Cover and J. A. Thomas. Elements of Information Theory. John Wiley & Sons, New York, NY, USA, 1991.

Digital Library

[4]

C. de Marcken. Unsupervised Language Acquisition. PhD thesis, Massachusetts Institute of Technology, 1996.

Digital Library

[5]

M. Farach-Colton. Personal communication.

[6]

T. Kida, Y. Shibata, M. Takeda, A. Shinohara, and S. Arikawa. A unifying framework for compressed pattern matching. In SPIRE/CRIWG, pages 89--96, 1999.

Digital Library

[7]

J. C. Kieffer and E. hui Yang. Grammar-based codes: A new class of universal lossless source codes. IEEE Transactions on Information Theory, 46(3):737--754, 2000.

Digital Library

[8]

J. C. Kieffer, E. hui Yang, G. J. Nelson, and P. Cosman. Universal lossless compression via multilevel pattern matching. IEEE Transactions on Information Theory, 46(4):1227--1245, 2000.

Digital Library

[9]

N. J. Larsson and A. Moffat. Offline dictionary-based compression. In Data Compression Conference, pages 296--305, 1999.

Digital Library

[10]

E. Lehman and A.Shelat. Approximations algorithms for grammar-based compression. In Thirteenth Annual Symposium on Discrete Algorithms (SODA'02), 2002.

Digital Library

[11]

C. Nevill-Manning. Inferring Sequential Structure. PhD thesis, University of Waikato, 1996.

[12]

C. G. Nevill-Manning and I. H. Witten. Compression and explanation using hierarchical grammars. The Computer Journal, 40(2/3):103--116, 1997.

[13]

J. A. Storer. Data Compression: Methods and Complexity Issues. PhD thesis, Princeton University, 1979.

Digital Library

[14]

E. h. Yang and J. C. Kieffer. Efficient universal lossless data compression algorithms based on a greedy sequential grammar transform--part one: Without context models. IEEE Transactions on Information Theory, 46(3):755--777, 2000.

Digital Library

[15]

J. Ziv and A. Lempel. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory, IT-23(3):337--343, 1977.

Digital Library

[16]

J. Ziv and A. Lempel. Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory, IT-24:530--536, 1978.

Cited By

Ryabko B(2024)Building Test Batteries Based on Analyzing Random Number Generator Tests within the Framework of Algorithmic Information TheoryEntropy10.3390/e2606051326:6(513)Online publication date: 14-Jun-2024
https://doi.org/10.3390/e26060513
Dyke Parunak H(2022)A Grammar-Based Behavioral Distance Measure Between Ransomware VariantsIEEE Transactions on Computational Social Systems10.1109/TCSS.2021.30609729:1(8-17)Online publication date: Feb-2022
https://doi.org/10.1109/TCSS.2021.3060972
Mondol TBrown D(2021)Grammar-based compression and its use in symbolic music analysisJournal of Mathematics and Music10.1080/17459737.2021.200295617:1(133-150)Online publication date: 6-Dec-2021
https://doi.org/10.1080/17459737.2021.2002956
Show More Cited By

Index Terms

Approximating the smallest grammar: Kolmogorov complexity in natural models

Recommendations

The Smallest Grammar Problem Revisited
In a seminal paper, Charikar <italic>et al.</italic> derive upper and lower bounds on the approximation ratios for several grammar-based compressors, but in all cases there is a gap between the lower and upper bound. Here the gaps for <italic>LZ78</italic>...
A really simple approximation of smallest grammar

In this paper we present a really simple linear-time algorithm constructing a context-free grammar of size 4 g log 3 / 2 ( N / g ) for the input string, where N is the size of the input string and g the size of the optimal grammar generating this ...
The smallest grammar problem

This paper addresses the smallest grammar problem: What is the smallest context-free grammar that generates exactly one given string σ? This is a natural question about a fundamental object connected to many fields such as data compression, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

STOC '02: Proceedings of the thiry-fourth annual ACM symposium on Theory of computing

May 2002

840 pages

ISBN:1581134959

DOI:10.1145/509907

Conference Chair:
John Reif
Duke University

Copyright © 2002 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 May 2002

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

STOC02

Sponsor:

SIGACT

STOC02: Symposium on the Theory of Computing

May 19 - 21, 2002

Quebec, Montreal, Canada

Acceptance Rates

STOC '02 Paper Acceptance Rate 91 of 287 submissions, 32%;

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Upcoming Conference

STOC '25

Sponsor:
sigact

57th Annual ACM Symposium on Theory of Computing (STOC 2025)

June 23 - 27, 2025

Prague , Czech Republic

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
78
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ryabko B(2024)Building Test Batteries Based on Analyzing Random Number Generator Tests within the Framework of Algorithmic Information TheoryEntropy10.3390/e2606051326:6(513)Online publication date: 14-Jun-2024
https://doi.org/10.3390/e26060513
Dyke Parunak H(2022)A Grammar-Based Behavioral Distance Measure Between Ransomware VariantsIEEE Transactions on Computational Social Systems10.1109/TCSS.2021.30609729:1(8-17)Online publication date: Feb-2022
https://doi.org/10.1109/TCSS.2021.3060972
Mondol TBrown D(2021)Grammar-based compression and its use in symbolic music analysisJournal of Mathematics and Music10.1080/17459737.2021.200295617:1(133-150)Online publication date: 6-Dec-2021
https://doi.org/10.1080/17459737.2021.2002956
Bao FGutierrez CJn Charles-Blount JYan YZhang Y(2018)Accelerating Boolean Satisfiability (SAT) solving by common subclause eliminationArtificial Intelligence Review10.1007/s10462-016-9530-649:3(439-453)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1007/s10462-016-9530-6
Gadde RMarlet RParagios N(2016)Learning Grammars for Architecture-Specific Facade ParsingInternational Journal of Computer Vision10.1007/s11263-016-0887-4117:3(290-316)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1007/s11263-016-0887-4
(2013)Performing Collective Tasks with Flagellated Bacteria Acting as Natural and Hybrid MicrorobotsHandbook of Collective Robotics10.1201/b14908-27(797-822)Online publication date: 29-May-2013
https://doi.org/10.1201/b14908-27
Claude FNavarro G(2012)Improved grammar-based compressed indexesProceedings of the 19th international conference on String Processing and Information Retrieval10.1007/978-3-642-34109-0_19(180-192)Online publication date: 21-Oct-2012
https://dl.acm.org/doi/10.1007/978-3-642-34109-0_19
Demaine EEisenstat SIshaque MWinslow A(2011)One-dimensional staged self-assemblyProceedings of the 17th international conference on DNA computing and molecular programming10.5555/2042033.2042045(100-114)Online publication date: 19-Sep-2011
https://dl.acm.org/doi/10.5555/2042033.2042045
Gawrychowski P(2011)Pattern matching in lempel-Ziv compressed stringsProceedings of the 19th European conference on Algorithms10.5555/2040572.2040619(421-432)Online publication date: 5-Sep-2011
https://dl.acm.org/doi/10.5555/2040572.2040619
Gawrychowski P(2011)Pattern Matching in Lempel-Ziv Compressed Strings: Fast, Simple, and DeterministicAlgorithms – ESA 201110.1007/978-3-642-23719-5_36(421-432)Online publication date: 2011
https://doi.org/10.1007/978-3-642-23719-5_36
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents