More Web Proxy on the site http://driver.im/

article

Recognition and retrieval of mathematical expressions

Authors:

Richard Zanibbi,

Dorothea BlosteinAuthors Info & Claims

International Journal on Document Analysis and Recognition, Volume 15, Issue 4

Pages 331 - 357

https://doi.org/10.1007/s10032-011-0174-4

Published: 01 December 2012 Publication History

Abstract

Document recognition and retrieval technologies complement one another, providing improved access to increasingly large document collections. While recognition and retrieval of textual information is fairly mature, with wide-spread availability of optical character recognition and text-based search engines, recognition and retrieval of graphics such as images, figures, tables, diagrams, and mathematical expressions are in comparatively early stages of research. This paper surveys the state of the art in recognition and retrieval of mathematical expressions, organized around four key problems in math retrieval (query construction, normalization, indexing, and relevance feedback), and four key problems in math recognition (detecting expressions, detecting and classifying symbols, analyzing symbol layout, and constructing a representation of meaning). Of special interest is the machine learning problem of jointly optimizing the component algorithms in a math recognition system, and developing effective indexing, retrieval and relevance feedback algorithms for math retrieval. Another important open problem is developing user interfaces that seamlessly integrate recognition and retrieval. Activity in these important research areas is increasing, in part because math notation provides an excellent domain for studying problems common to many document and graphics recognition and retrieval applications, and also because mature applications will likely provide substantial benefits for education, research, and mathematical literacy.

References

[1]

Adeel M., Cheung H.S., Khiyal H.S.: Math go! prototype of a content based mathematical formula search engine. J. Theor. Appl. Inf. Technol. 4(10), 1002---1012 (2008)

[2]

Aho A.V., Kernighan B.W., Weinberger P.J.: The AWK Programming Language. Addison-Wesley, New York (1988)

[3]

Altamimi, M., Youssef, A.S.: An extensive math query language. In: ISCA International Conference on Software Engineering and Data Engineering, pp. 57---63. Las Vegas, USA (2007)

[4]

Aly W., Uchida S., Suzuki M.: Identifying subscripts and superscripts in mathematical documents. Math. Comput. Sci. 2(2), 195---209 (2008)

[5]

Anderson, R.H.: Syntax-Directed Recognition of Hand-Printed Two-Dimensional Equations. PhD thesis, Harvard University, Cambridge (1968)

[6]

Anderson R.H.: Two-dimensional mathematical notation. In: Fu, K.S. (ed.) Syntactic Pattern Recognition, Applications, pp. 174---177. Springer, New York (1977)

[7]

Anthony, L., Yang, J., Koedinger, K.R.: Adapting handwriting recognition for applications in algebra learning. In: Proceedings of the ACM Workshop on Educational Multimedia and Multimedia Education, pp. 47---56. Augsburg (2007)

Digital Library

[8]

Ashida, K., Okamoto, M., Imai, H., Nakatsuka, T.: Performance evaluation of a mathematical formula recognition system with a large scale of printed formula images. In: Proceedings of the International Conference on Document Image Analysis for Libraries, pp. 320---331. Lyon (2006)

Digital Library

[9]

Asperti, A., Guidi, F., Coen, C., Tassi, E., Zacchiroli, S.: A content based mathematical search engine: whelp. In: Proceedings of Types for Proofs and Programs 2004, vol. 3839 of LNCS, pp. 17---32. Springer (2006)

[10]

Ausbrooks, R., Buswell, S., Carlisle, D., Chavchanidze, G., Dalmas, S., Devitt, S., Diaz, A., Dooley, S., Hunter, R., Ion, P., Kohlhase, M., Lazrek, A., Libbrecht, P., Miller, B., Miner, R., Rowley, C., Saregent, M., Smith, B., Soiffer, N., Sutor, R., Watt, S. (2010) Mathematical markup language (MathML) version 3.0, W3C recommendation (http://www.w3.org/math/)

[11]

Awal, A.M., Mouchère, H., Viard-Gaudin, C.: Towards handwritten mathematical expression recognition. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1046---1050. Barcelona (2009)

[12]

Awal, A.M., Mouchére, H., Viard-Gaudin, C.: The problem of handwritten mathematical expression recognition evaluation. In: Proceedings of the International Conference on Frontiers in Handwriting Recognition, pp. 646---651. Montréal (2010)

[13]

Baker, J.B., Sexton, A.P., Sorge, V.: A linear grammar approach to mathematical formula recognition from PDF. In: Proceedings of Mathematical Knowledge Management, vol. 5625 of LNAI, pp. 201---216. Springer (2009)

[14]

Baker, J.B., Sexton, A.P., Sorge, V.: Faithful mathematical formula recognition from PDF documents. In: Proceedings of the International Workshop on Document Analysis Systems, pp. 485---492, Boston (2010)

Digital Library

[15]

Belongie S., Malik J., Puzicha J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 509---522 (2002)

Digital Library

[16]

Berman, B.P., Fateman, R.J.: Optical character recognition for typeset mathematics. In: Proceedings of the International Syposium on Symbolic and Algebraic Computation, pp. 348---353. Oxford (1994)

[17]

Bernstein, M.I.: Computer input/output of two-dimensional notations. In: Proceedings of the Syposium on Symbolic and Algebraic Manipulation, pp. 102---103 (1971)

[18]

Blostein, D., Cordy, J., Zanibbi, R.: Applying compiler techniques to diagram recognition. In: Proceedings of the International Conference on Pattern Recognition, vol. 3, pp. 123---126. Québec (2002)

[19]

Blostein, D., Grbavec, A.: Recognition of mathematical notation. In: Handbook of Character Recognition and Document Image Analysis, pp. 557---582. World Scientific (1997)

[20]

Blostein, D., Lank, E., Rose, A., Zanibbi, R.: User interfaces for on-line diagram recognition. In: Selected Papers from the Fourth International Workshop on Graphics Recognition Algorithms and Applications, vol. 2390 of LNCS, pp. 92---103. Springer (2002)

[21]

Blostein, D., Lank, E., Zanibbi, R.: Treatment of diagrams in document image analysis. In: Proceedings of the International Conference on Theory and Application of Diagrams, pp. 330---344. Springer, London (2000)

Digital Library

[22]

Borlund, P.: User-centered evaluation of information retrieval systems. In: Information Retrieval: searching in the 21st Century, pp. 21---37. Wiley (2009)

[23]

Bunt, A., Terry, M., Lank, E.: Friend or foe? Examining CAS use in mathematics research. In: Proceedings of the International Conference on Human Factors in Computing Systems, pp. 229---238. New York (2009)

[24]

Cajori F.: A History of Mathematical Notations. 2 vols. Open Court Publishing Company, Chicago (1929)

[25]

Carette, J., Farmer, W.M.: A review of mathematical knowledge management. In: Proceedings of Mathematical Knowledge Management, vol. 5625 of LNAI, pp. 233---246. Springer (2009)

[26]

Case D.O.: Looking for Information: a Survey of Research on Information Seeking, Needs, and Behavior. Academic Press, San Diego (2002)

[27]

Casey R.G., Lecolinet E.: A survey of methods and strategies in character segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18(7), 690---706 (1996)

Digital Library

[28]

Chan K.-F., Yeung D.-Y.: Mathematical expression recognition: a survey. Int. J. Doc. Anal. Recognit. 3, 3---15 (2000)

[29]

Chan K.-F., Yeung D.-Y.: Error detection, error correction and performance evaluation in on-line mathematical expression recognition. Pattern Recognit. 34(8), 1671---1684 (2001a)

[30]

Chan, K.-F., Yeung, D.-Y.: Pencalc: a novel application of on-line mathematical expression recognition technology. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 774---778. Seattle (2001b)

[31]

Chang S.-K.: A method for the structural analysis of two-dimensional mathematical expressions. Inf. Sci. 2, 253---272 (1970)

Digital Library

[32]

Char, B.W., Watt, S.M.: Representing and characterizing handwritten mathematical symbols through succinct functional approximation. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1198---1202. Curitiba (2007)

[33]

Chaundy T.W., Barrett P.R., Batey Charles: The Printing of Mathematics. Oxford University Press, London (1957)

[34]

Chou, P.A.: Recognition of equations using a two-dimensional stochastic context-free grammar. In: Proceedings of Visual Communications and Image Processing IV, vol. 1199 of Proceeding SPIE, pp. 852---863 (1989)

[35]

Datta R., Joshi D., Li J., Wang J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(2), 1---60 (2008)

Digital Library

[36]

Davenport, J.H., Kohlhase, M.: Unifying math ontologies: a tale of two standards. In: Intelligent Computer Mathematics, vol. 5625 of LNAI, pp. 263---278. Springer (2009)

Digital Library

[37]

Dewar M.: Openmath: an overview. ACM SIGSAM Bull. 34, 2---5 (2000)

Digital Library

[38]

Doermann D.: The indexing and retrieval of document images: a survey. J. Comput. Vis. Imag. Underst. 70, 287---298 (1998)

Digital Library

[39]

Drake, D.M., Baird, H.S.: Distinguishing mathematics notation from english text using computational geometry. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1270---1274. Seoul (2005)

[40]

Duda R.O., Hart P.E., Stork D.G.: Pattern Classification. 2nd edn. Springer, New York (2001)

[41]

Einwohner, T.H., Fateman, R.J.: Searching techniques for integral tables. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation, pp. 133---139. Montréal (1995)

[42]

Eto, Y., Suzuki, M.: Mathematical formula recognition using virtual link network. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 430---437. Seattle (2001)

[43]

Fateman, R.J., Tokuyasu, T.: Progress in recognizing typeset mathematics. In: Proceedings of the SPIE, vol. 2660, pp. 37---50 (1996)

[44]

Fitzgerald, J.A., Geiselbrechtinger, F., Kechadi, T.: Mathpad: a fuzzy logic-based recognition system for handwritten mathematics. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 694---698. Curitiba (2007)

[45]

Freund Y., Schapire R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119---139 (1995)

Digital Library

[46]

Fujimoto, M., Kanahori, T., Suzuki, M.: Infty editor--a mathematics typesetting tool with a handwriting interface and a graphical front-end to OpenXM servers. In: Computer Algebra-- Algorithms, Implementations and Applications, vol. 1335 of RIMS Kokyuroku, pp. 217---226 (2003)

[47]

Fujiyoshi, A., Suzuki, M., Uchida, S.: Verification of mathematical formulae based on a combination of context-free grammar and tree grammar. In: Proceedings of the International Conference on Mathematical Knowledge Management, vol. 5144 of LNCS, pp. 415---429. Springer (2008)

[48]

Fujiyoshi, A., Suzuki, M., Uchida, S.: Syntactic detection and correction of misrecognitions in mathematical OCR. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1360---1364. Barcelona (2009)

[49]

Garain, U.: Identification of mathematical expressions in document images. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1340---1344. Barcelona (2009)

[50]

Garain U., Chaudhuri B.B.: Recognition of online handwritten mathematical expressions. IEEE Trans. Syst. Man Cybern. Part B 34(6), 2366---2376 (2004)

Digital Library

[51]

Garain U., Chaudhuri B.B.: A corpus for OCR research on mathematical expressions. Int. J. Doc. Anal. Recognit. 7(4), 241---259 (2005)

Digital Library

[52]

Garain, U., Chaudhuri, B.B.: OCR of printed mathematical expressions. In: Digital Document Processing, pp. 235---259. Springer (2007)

[53]

Genoe, R., Fitzgerald, J.A., Kechadi, T.: An online fuzzy approach to the structural analysis of handwritten mathematical expressions. In: Proceedings of the International Conference on Fuzzy Systems, pp. 242---250. Vancouver (2006)

[54]

Golubitsky, O., Watt, S.M.: Online computation of similarity between handwritten characters. In: Proceedings of Document Recognition and Retrieval, vol. 7247 of Proceedings of the SPIE, pp. C1---C10, San Jose (2009)

[55]

Golubitsky O., Watt S.M.: Distance-based classification of handwritten symbols. Int. J. Doc. Anal. Recognit. 13(2), 133---146 (2010)

Digital Library

[56]

Golubitsky, O., Watt, S.M.: Improved classification through runoff elections. In: Proceeding of the Workshop on Document Analysis Systems, pp. 59---64. Boston (2010)

[57]

Graf, P.: Substitution tree indexing. In: Proceedings of the International Conference on Rewriting Techniques and Applications, pp. 117---131. London (1995)

[58]

Grbavec, A., Blostein, D.: Mathematics recognition using graph rewriting. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 417---421. Montréal (1995)

[59]

Hashimoto, H., Hijikata, Y., Nishida, S.: Incorporating breadth first search for indexing MathML objects. In: Proceedings of the International Conference on Systems, Man and Cybernetics, pp. 3519---3523, Singapore (2008)

[60]

Hatcher E., Gospodnetić O.: Lucene in Action. 2nd edn. Manning, USA (2010)

[61]

Hearst M.A.: Search User Interfaces. 1st edn. Cambridge University Press, New York (2009)

[62]

Hiemstra, D.: Information retrieval models. In: Information Retrieval: searching in the 21st Century, pp. 1---17. Wiley (2009)

[63]

Higham N.J.: Handbook of Writing for the Mathematical Sciences. Society for Industrial and Applied Mathematics, Philadelphia (1993)

[64]

Hu J., Kashi R.S., Lopresti D., Wilfong G.T.: Evaluating the performance of table processing algorithms. Int. J. Doc. Anal. Recognit. 4(3), 140---153 (2002)

[65]

Hull, J.F.: Recognition of Mathematics Using a Two-Dimensional Trainable Context-Free Frammar. Master's thesis, MIT, Cambridge (1996)

[66]

Kacem A., Belaid A., Ben Ahmed M.: Automatic extraction of printed mathematical formulas using fuzzy logic and propagation of context. Int. J. Doc. Anal. Recognit. 4, 97---108 (2001)

[67]

Kamali, S., Tompa, F.: Improving mathematics retrieval. In: Proceedings of Digital Mathematics Libraries, pp. 37---48, Grand Bend (2009)

[68]

Kanahori, T., Sexton, A.P., Sorge, V., Suzuki, M.: Capturing abstract matrices from paper. In: Borwein, J.M., Farmer, W.M. (eds.) Proceedings of Mathematical Knowledge Management, vol. 4108 of LNAI, pp. 124---138. Springer (2006)

[69]

Kanahori, T., Suzuki, M.: A recognition method of matrices by using variable block pattern elements generating rectangular areas. In: Graphics Recognition--Algorithms and Applications, vol. 2390 of LNCS, pp. 320---329. Springer (2002)

[70]

Kanahori, T., Suzuki, M.: Detection of matrices and segmentation of matrix elements in scanned images of scientific documents. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 433---437. Edinburgh (2003)

[71]

Kanahori, T., Suzuki, M.: Refinement of digitized documents through recognition of mathematical formulae. In: Proceedings of the International Workshop on Document Image Analysis for Libraries, pp. 27---28. Lyon (2006)

[72]

Kanungo T., Haralick R.M., Baird H.S., Stuetzle W., Madigan D.: A statistical, nonparametric methodology for document degradation model validation. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1209---1223 (2000)

Digital Library

[73]

Kim, K., Rhee, T.-H., Lee, J.S., Kim, J.H.: Utilizing consistency context for handwritten mathematical expression recognition. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1051---1055. Barcelona (2009)

[74]

Knuth, Donald E.: Tex and METAFONT--New Directions in Typesetting. Digital Press, Bedford (1979)

[75]

Kohlhase, A., Kohlhase, M.: Re-examining the MKM value proposition: from math web search to math web research. In: Proceedings of the Symposium on Towards Mechanized Mathematical Assistants, vol. 4573 of LNCS, pp. 313---326. Springer (2007)

[76]

Kohlhase, M.: OMDoc: an Open Markup Format for Mathematical Documents, vol. 4180 of LNAI. Springer (2006)

[77]

Kohlhase, M., Anca, S., Jucovschi, C., Palomo, A.G., Sucan, I.: MathWebSearch 0.4: a semantic search engine for mathematics. (unpublished manuscript. http://kwarc/info/kohlhase/publications.html) (2008)

[78]

Kohlhase, M., Sucan, I.: A search engine for mathematical formulae. In: Proc. Artificial Intelligence and Symbolic Computation, volume 4120 of LNAI, pp. 241---253. Springer (2006)

[79]

Kopec G.E., Chou P.A.: Document image decoding using Markov source models. IEEE Trans. Pattern Anal. Mach. Intell. 16(6), 602---617 (1994)

Digital Library

[80]

Kosmala, A., Rigoll, G.: On-line handwritten formula recognition using statistical methods. In: Proceedings of the International Conference on Pattern Recognition, pp. 1306---1308. Brisbane (1998)

[81]

Labahn, G., Lank, E., MacLean, S., Marzouk, M., Tausky, D.: Mathbrush: a system for doing math on pen-based devices. In: Proceedings of the Workshop on Document Analysis Systems, pp. 599---606. Japan (2008)

[82]

Labahn, G., Lank, E., Marzouk, M., Bunt, A., MacLean, S., Tausky, D.: Mathbrush: a case study for pen-based interactive mathematics. In: Proceedings of Eurographics Workshop Sketch-Based Interfaces and Modeling, Annecy. France (2008)

[83]

Lapointe, A., Blostein, D.: Issues in performance evaluation: A case study of math recognition. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1355---1359. Barcelona (2009)

[84]

LaViola, J.J., Leal, A., Miller, T.S., Zeleznik, R.C.: Evaluation of techniques for visualizing mathematical expression recognition results. In: Proceedings of Graphics Interface, pp. 131---138. Windsor, Canada (2008)

[85]

La Viola J.J., Zeleznik R.C.: Mathpad2: a system for the creation and exploration of mathematical sketches. ACM Trans. Graph. 23(3), 432---440 (2004)

[86]

La Viola J.J., Zeleznik R.C.: A practical approach to writer- dependent symbol recognition using a writer-independent recognizer. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1917---1926 (2007)

[87]

Lavirotte, S., Pottier, L.: Optical formula recognition. In: Proceedings of the International Conference on Document Analysis and Recognition, vol. 1, pp. 357---361. Ulm (1997)

[88]

Lee, H.-J., Wang, J.-S.: Design of a mathematical expression understanding system. Pattern Recognit. Lett. 18(3), 289---298 (1997)

Digital Library

[89]

Li, C., Zeleznik, R.C., Miller, T., La Viola, J.J.: Online recognition of handwritten mathematical expressions with support for matrices. In: Proceedings of the International Conference on Pattern Recognition, pp. 1---4. Tampa (2008)

[90]

Liang, P., Narasimhan, M., Shilman, M., Viola, P.A.: Efficient geometric algorithms for parsing in two dimensions. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1172---1177. Seoul (2005)

[91]

Libbrecht, P., Melis, E.: Methods for access and retrieval of mathematical content in ActiveMath. In: Proceedings of the International Congress on Mathematical Software, vol. 4151 of LNCS, pp. 331---342. Springer (2006)

[92]

Lopresti, D., Wilfong, G.: Evaluating document analysis results via graph probing. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 116---120. Seattle (2001)

[93]

MacLean S., Labahn G., Lank E., Marzouk M., Tausky D.: Grammar-based techniques for creating ground-truthed sketch corpora. Int. J. Doc. Anal. Recognit. 14(1), 65---74 (2011)

Digital Library

[94]

Malon C.D., Uchida S., Suzuki M.: Mathematical symbol recognition with support vector machines. Pattern Recognit. Lett. 29, 1326---1332 (2008)

Digital Library

[95]

Manning C.D., Raghavan P., Schütze H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)

[96]

Marinai, S., Miotti, B., Soda, G.: Mathematical symbol indexing using topologically ordered clusters of shape contexts. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1041---1045. Barcelona (2009)

[97]

Marriott, K., Meyer, B., Wittenburg, K.D.: A survey of visual language specification and recognition. In: Visual Language Theory, pp. 5---85. Springer (1998)

Digital Library

[98]

Martin, W.A.: Computer input/output of mathematical expressions. In: Proceedings of the Symposium on Symbolic and Algebraic Manipulation, pp. 78---89. Los Angeles (1971)

[99]

Mathematical Reviews and Zentralblatt für Mathematik. Mathematics subject classification. http://www.ams.org/mathscinet/msc/msc2010.html (2010)

[100]

Matsakis, N.: Recognition of Handwritten Mathematical Expressions. Master's thesis, MIT, Cambridge (1999)

[101]

Michler G.O.: Report on the retrodigitization project "Archiv der Mathematik. Archiv der Mathematik 77, 116---128 (2001)

[102]

Michler G.O.: How to build a prototype for a distributed digital mathematics archive library. Ann. Math. Artif. Intell. 38, 137---164 (2003)

Digital Library

[103]

Miller B.R., Youssef A.S.: Technical aspects of the digital library of mathematical functions. Ann. Math. Artif. Intell. 38, 121---136 (2003)

[104]

Miller, E.G., Viola, P.A.: Ambiguity and constraint in mathematical expression recognition. In: Proceedings of the 15th National Conference on Artificial Intelligence, pp. 784---791. Madison (1998)

Digital Library

[105]

Miner, R., Munavalli, R.: An approach to mathematical search through query formulation and data normalization. In: Towards Mechanized Mathematical Assistants, vol. 4573 of LNAI, pp. 342---355. Springer (2007)

[106]

Miyazaki, Y., Iguchi, Y.: Development of information-retrieval tool for MathML-based math expressions. In: Proceedings of the International Conference on Computers in Education, pp. 419---426. Tapei (2008)

[107]

Munavalli, R., Miner, R.: Mathfind: a math-aware search engine. In: Proceedings of the International Conference on Information Retrieval, pp. 735---735. New York (2006)

[108]

Nagy G.: Twenty years of document image analysis in PAMI. IEEE Trans on Pattern Anal. Mach. Intell. 22(1), 38---62 (2000)

Digital Library

[109]

Nagy, G., Seth, S.: Hierarchical representation of optically scanned documents. In: Proceedings of International Conference on Pattern Recognition, pp. 347---349. Montréal (1984)

[110]

Normann, I., Kohlhase, M.: Extended formula normalization for $${\epsilon}$$ -retrieval and sharing of mathematical knowledge. In: Proceedings of Towards Mechanized Mathematical Assistants, vol. 4573 of LNAI, pp. 356---370. Springer (2007)

[111]

Okamoto, M., Imait, K.T.: Performance evaluation of a robust method for mathematical expression recognition. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 121---128. Seattle, (2001)

[112]

Okamoto, M., Miao, B.: Recognition of mathematical expressions by using the layout structures of symbols. In: Proceedings of the International Conference on Document Analysis and Recognition, vol. 1, pp. 242---250. Saint-Malo (1991)

[113]

Okamoto, M., Miyazawa, A.: An experimental implementation of a document recognition system for papers containing mathematical expressions. In: Structured Document Image Analysis, pp. 36---53. Springer (1992)

[114]

Panic, M.: Math handwriting recognition in Windows 7 and its benefits. In: Intelligent Computer Mathematics, vol. 5625 of LNCS, pp. 29---30. Springer (2009)

[115]

Phillips, I.: Methodologies for using UW databases for OCR and image understanding systems. In: Proceedings of Document Recognition V, vol. 3305 of SPIE Proceedings, pp. 112---127. San Jose (1998)

[116]

Plamandon R., Srihari S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63---84 (2000)

[117]

Pollanen, M., Wisniewski, T., Yu, X.: Xpress: a novice interface for the real-time communication of mathematical expressions. In: Proceedings of the Workshop on Mathematical User-Interfaces. Linz (2007)

[118]

Rabiner L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257---286 (1989)

[119]

Raja, A., Rayner, M., Sexton, A.P., Sorge, V.: Towards a parser for mathematical formula recognition. In: Mathematical Knowledge Management, vol. 4108 of LNAI, pp. 139---151. Springer (2006)

[120]

Rath, T.M., Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of Computer Vision and Pattern Recognition, pp. 521---527. Madison, (2003)

[121]

Rath T.M., Manmatha R.: Word spotting for historical documents. Int. J. Doc. Anal. Recognit. 9, 139---152 (2007)

Digital Library

[122]

Rhee T.H., Kim J.H.: Efficient search strategy in structural analysis for handwritten mathematical expression recognition. Pattern Recognit. 42(12), 3192---3201 (2009)

Digital Library

[123]

Rüger, S.: Multimedia resource discovery. In: Information Retrieval: Searching in the 21st Century, pp. 39---62. Wiley (2009)

[124]

Sain K., Dasgupta A., Garain U.: EMERS: A tree matching-based performance evaluation of mathematical expression recognition systems. Int. J. Doc. Anal. Recognit. 14(1), 75---85 (2011)

Digital Library

[125]

Salton G., McGill M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)

[126]

Samet H.: Foundations of Multidimensional and Metric Data Structures. Morgan Kaufmann, New York (2006)

[127]

Sexton, A.P., Sorge, V.: Abstract matrices in symbolic computation. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation, pp. 318---325. Genoa (2006)

[128]

Shafait F., Keysers D., Breuel T.M.: Performance evaluation and benchmarking of six page segmentation algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 941---954 (2008)

Digital Library

[129]

Shatnawi, M., Youssef, A.S.: Equivalence detection using parse-tree normalization for math search. In: Proceedings of the International Conference on Digital Information Management, vol. 2, pp. 643---648. Lyon (2007)

[130]

Shi, Y., Li, H.Y., Soong, F.K.: A unified framework for symbol segmentation and recognition of handwritten mathematical expressions. In: Proceedings of the International Conference on Document Analysis and Recognition, vol. 2, pp. 854---858. Curitiba (2007)

[131]

Shi, Y., Soong, F.K.: Symbol graph based discriminative training and rescoring for improved math symbol recognition. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pp. 1953---1956. Las Vegas (2008)

[132]

Smeulders A.W.M., Worring M., Santini S., Gupta A., Jain R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349---1380 (2000)

Digital Library

[133]

Smirnova, E., Watt, S.M.: Communicating mathematics via pen-based computer interfaces. In: Proceedings of the International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, pp. 9---18. Timişoara (2008)

[134]

Smirnova, E., Watt, S.M.: Context-sensitive mathematical character recognition. In: Proceedings of the International Conference on Frontiers in Handwriting Recognition, pp. 604---610. Montréal (2008)

[135]

Smithies, S.: Freehand Formula Entry System. Master's thesis, University of Otago, Dunedin (1999)

[136]

Smithies S.: Equation entry and editing via handwriting and gesture recognition. Beh. Inf. Technol. 20(1), 53---67 (2001)

[137]

Smithies, S., Novins, K., Arvo, J.: A handwriting-based equation editor. In: Proceedings of Graphics Interface, pp. 84---91. Kingston (1999)

[138]

So, C.M., Watt, S.M.: Determining empirical charateristivs of mathematical expression use. In: Proceedings of Mathematical Knowledge Management, vol. 3863 of LNCS, pp. 361---375. Springer (2005)

[139]

So, C.M., Watt, S.M.: On the conversion between content MathML and OpenMath. In: Proceedings of the Conference on Communicating Mathematics in the Digital Era, pp. 169---182. Aveiro (2006)

[140]

Suzuki, M., Kanahori, T., Ohtake, N., Yamaguchi, K.: An integrated OCR software for mathematical documents and its output with accessibility. In: Proceedings of the International Conference on Computers Helping People with Special Needs, vol. 3119 of LNCS, pp. 648---655. Springer (2004)

[141]

Suzuki, M., Tamari, F., Fukuda, R., Uchida, S., Kanahori, T.: INFTY: An integrated OCR system for mathematical documents. In: Proceedings of Document Engineering, pp. 95---104. Grenoble (2003)

[142]

Suzuki, M., Uchida, S., Nomura, A.: A ground-truthed mathematical character and symbol image database. In: Proceedings of the International Conference on Document Analysis and Recognition, vol. 2, pp. 675---679. Seoul (2005)

[143]

Takiguchi, Y., Okada, M., Miyake, Y.: A fundamental study of output translation from layout recognition and semantic understanding system for mathematical formulae. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 745---749. Seoul (2005)

[144]

Tapia, E., Rojas, R.: Recognition of on-line handwritten mathematical formulas in the e-chalk system. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 980---984. Edinburgh (2003)

[145]

Tapia, E., Rojas, R.: Recognition of on-line handwritten mathematical expressions using a minimum spanning tree construction and symbol dominance. In: Graphics Recognition: Recent Advances and Persepectives, vol. 3088 of LNCS, pp. 329---340. Springer (2004)

[146]

Tapia, E., Rojas, R.: A survey on recognition of on-line handwritten mathematical notation. Technical Report B-07-01, Free University of Berlin (2007)

[147]

Tausky, D., Labahn, G., Lank, E., Marzouk, M.: Managing ambiguity in mathematical matrices. In: Proceedings of the Eurographics Workshop on Sketch-Based Interfaces and Modeling, pp. 115---122. Riverside (2007)

Digital Library

[148]

The OpenMath Society. http://www.openmath.org/

[149]

Toussaint G.T.: The use of context in pattern recognition. Pattern Recognit. 10, 189---204 (1978)

[150]

Toyozumi, K., Suzuki, T., Mori, J., Suenaga, Y.: A system for real-time recognition of handwritten mathematical formulas. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 1059---1063. Seattle (2001)

[151]

Toyozumi K., Takahiro S., Mori K., Suenaga Y.: An on-line handwritten mathemical equation recognition system that can process matrix expressions by referring to the relative positions of matrix elements. Syst. Comput. Jpn. 37(14), 87---96 (2006)

Digital Library

[152]

Toyozumi, K., Yamada, N., Mase, K., Kitasaka, T., Mori, K., Suenaga, Y., Takahashi, T.: A study of symbol segmentation method for handwritten mathematical formula recognition using mathematical structure information. In: Proceedings of the International Conference on Pattern Recognition, vol. 2, pp. 630---633. Cambridge (2004)

[153]

Twaakyondo, H.M., Okamoto, M.: Structure analysis and recognition of mathematical expressions. In: Proceedings of the International Conference on Document Analysis and Recognition, vol. 1, pp. 430---437. Montréal (1995)

[154]

Vapnik V.: The Nature of Statistical Learning Theory. Springer, New York (1995)

[155]

Watt, S.M.: An empirical measure on the set of symbols occurring in engineering mathematics texts. In: Proceedings of the International Workshop on Document Analysis Systems, pp. 557---564. Japan (2008)

[156]

Westman, S.: Image users' needs and searching behaviour. In: Information Retrieval: searching in the 21st Century, pp. 63---83. Wiley (2009)

[157]

Wick, K.: Rules for Typesetting Mathematics Czechoslovak Academy of Sciences. The Hague (1965). Translated by V. Boublik and M. Hejlova

[158]

Winkler, H.-J.: HMM-based handwritten symbol recognition using on-line and off-line features. In: Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing, pp. 3438---3441. Atlanta (1996)

[159]

Yamamoto, R., Sako, S., Nishimoto, T., Sagayama, S.: On-line recognition of handwritten mathematical expressions based on stroke-based stochastic context-free grammar. In: Proceedings of the International Workshop on Frontiers in Handwriting Recognition, pp. 249---254, La Baule (2006)

[160]

Yokoi, K., Aizawa, A.: An approach to similarity search for mathematical expressions using MathML. In: Proceedings of Digital Mathematics Libraries, pp. 27---35. Grand Bend (2009)

[161]

Yu, Li: Image-Based Math Retrieval Using Handwritten Queries. Master's thesis, Rochester Institute of Technology, Rochester (2010)

[162]

Zanibbi, R., Blostein, D., Cordy, J.R.: Baseline structure analysis of handwritten mathematics notation. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 768---773. Seattle (2001)

[163]

Zanibbi R., Blostein D., Cordy J.R.: Recognizing mathematical expressions using tree transformation. IEEE Trans. Pattern Anal. Mach. Intell. 24(11), 1455---1467 (2002)

Digital Library

[164]

Zanibbi, R., Blostein, D., Cordy, J.R.: Recognition tasks are imitation games. In: LNCS, vol. 3686, pp. 209---218 (2005)

[165]

Zanibbi, R., Novins, K., Arvo, J., Zanibbi, K.: Aiding manipulation of handwritten mathematical expressions through style- preserving morphs. In: Proceedings of Graphics Interface, pp. 127---134. Ottawa (2001)

[166]

Zanibbi, R., Pillay, A., Mouchére, H., Viard-Gaudin, C., Blostein, D.: Stroke-based performance metrics for handwritten mathematical expressions. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 334---338. Beijing (2011)

[167]

Zanibbi, R., Yu, L.: Math spotting: Retrieving math in technical documents using handwritten query images. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 446---451. Beijing (2011)

[168]

Zanibbi, R., Yuan, B.: Keyword and image-based retrieval for mathematical expressions. In: Proceedings of Document Recognition and Retrieval XVIII, vol. 7874 of SPIE Proceedings, paper OI, pp. 1---9. San Francisco (2011)

[169]

Zeleznik, R.C., Miller, T., Li, C., La Viola, J.J.: Mathpaper: mathematical sketching with fluid support for interactive computation. In: International Symposium on Smart Graphics, vol. 5166 of LNCS, pp. 20---32. Springer (2008)

[170]

Zhang, L., Blostein, D., Zanibbi, R.: Using fuzzy logic to analyze superscript and subscript relations in handwritten mathematical expressions. In: Proceedings of the International Conference on Document Analysis and Recognition, pp. 972---976. Seoul, Korea (2005)

[171]

Zhao, J., Kan, M.-Y., Theng, Y.L.: Math information retrieval: user requirements and prototype implementation. In: Proceedings of the ACM/IEEE Joint Conference Digital libraries, pp. 187---196. New York (2008)

[172]

Zhou X.S., Huang T.S.: Relevance feedback in image retrieval: a comprehensive review. Multimed. Syst. 8, 536---544 (2003)

[173]

Zipf G.K.: Human Behavior and the Principle of Least-Effort. Addison-Wesley, Reading (1949)

[174]

Zwillinger D.: CRC Standard Mathematical Tables and Formulae. CRC Press, Boca Raton (1996)

Cited By

Dadure PPakray PBandyopadhyay S(2024)Mathematical Information Retrieval: A ReviewACM Computing Surveys10.1145/369995357:3(1-34)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3699953
Lin CLin JLin SLiao J(2024)A Heuristic Algorithm for Mathematical Markup Encoding Based on the Relative Positions of CharactersProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685659(1-10)Online publication date: 20-Aug-2024
https://dl.acm.org/doi/10.1145/3685650.3685659
Ahadian PFeng YKosko KFerdig RGuan QLo DGamess E(2024)MNIST-Fraction: Enhancing Math Education with AI-Driven Fraction Detection and AnalysisProceedings of the 2024 ACM Southeast Conference10.1145/3603287.3651221(284-290)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3603287.3651221
Show More Cited By

Recognition and retrieval of mathematical expressions
1. Computing methodologies
2. Information systems

Recommendations

A Mathematical Information Retrieval System Based on RankBoost
JCDL '16: Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries

Mathematical Information Retrieval (MIR) systems are designed to help users to find related formulae and further understand the formulae in scientific documents. However, in existing MIR systems, nearly all the ranker models of MIR systems are based on ...
Multimedia retrieval by means of merge of results from textual and content based retrieval subsystems
CLEF'09: Proceedings of the 10th international conference on Cross-language evaluation forum: multimedia experiments

The main goal of this paper it is to present our experiments in ImageCLEF 2009 Campaign (photo retrieval task). In 2008 we proved empirically that the Text-based Image Retrieval (TBIR) methods defeats the Content-based Image Retrieval CBIR "quality" of ...
Interactive content-based image retrieval using relevance feedback

Database search engines are generally used in a one-shot fashion in which a user provides query information to the system and, in return, the system provides a number of database instances to the user. A relevance feedback system allows the user to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image International Journal on Document Analysis and Recognition

International Journal on Document Analysis and Recognition Volume 15, Issue 4

December 2012

101 pages

ISSN:1433-2833

EISSN:1433-2825

Issue’s Table of Contents

Copyright © Copyright © 2012 Springer-Verlag Berlin Heidelberg.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 December 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

77
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dadure PPakray PBandyopadhyay S(2024)Mathematical Information Retrieval: A ReviewACM Computing Surveys10.1145/369995357:3(1-34)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3699953
Lin CLin JLin SLiao J(2024)A Heuristic Algorithm for Mathematical Markup Encoding Based on the Relative Positions of CharactersProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685659(1-10)Online publication date: 20-Aug-2024
https://dl.acm.org/doi/10.1145/3685650.3685659
Ahadian PFeng YKosko KFerdig RGuan QLo DGamess E(2024)MNIST-Fraction: Enhancing Math Education with AI-Driven Fraction Detection and AnalysisProceedings of the 2024 ACM Southeast Conference10.1145/3603287.3651221(284-290)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3603287.3651221
Li ZWang XLiu YJin LHuang YDing K(2024)Improving Handwritten Mathematical Expression Recognition via Similar Symbol DistinguishingIEEE Transactions on Multimedia10.1109/TMM.2023.326064826(90-102)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3260648
Thakur USharma A(2024)Offline handwritten mathematical recognition using adversarial learning and transformersInternational Journal on Document Analysis and Recognition10.1007/s10032-023-00451-w27:2(147-158)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10032-023-00451-w
Ingram WWu JKahu SManzoor JBanerjee BAhuja AChoudhury MSalsabil LShields WFox E(2024)Building datasets to support information extraction and structure parsing from electronic theses and dissertationsInternational Journal on Digital Libraries10.1007/s00799-024-00395-425:2(175-196)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s00799-024-00395-4
Gao SNg Y(2023)Recommending Answers to Math Questions Based on KL-Divergence and Approximate XML Tree MatchingProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3624918.3625337(21-31)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3624918.3625337
Fu YCai WGao MZhou A(2023)Symbol Location-Aware Network for Improving Handwritten Mathematical Expression RecognitionProceedings of the 2023 ACM International Conference on Multimedia Retrieval10.1145/3591106.3592259(516-524)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3591106.3592259
Zhao XZhou KZhang BGong ZChen ZZhou YWen JSha JWang SLiu CHu GSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)JiuZhang 2.0: A Unified Chinese Pre-trained Language Model for Multi-task Mathematical Problem SolvingProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599850(5660-5672)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599850
Amador BLangsenkamp MDey AShah AZanibbi RChen HDuh WHuang HKato MMothe JPoblete B(2023)Searching the ACL Anthology with Math Formulas and TextProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591803(3110-3114)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591803
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents