Macsyma

The project was initiated in July, 1968 by Carl Engelman,^[2] William A. Martin (front end, expression display, polynomial arithmetic) and Joel Moses (simplifier, indefinite integration: heuristic/Risch). Martin was in charge of the project until 1971, and Moses ran it for the next decade. Engelman and his staff left in 1969 to return to The MITRE Corporation.^[1] Some code came from earlier work, notably Knut Korsvold's simplifier.^[3][4] Later major contributors to the core mathematics engine were:^[5] Yannis Avgoustis (special functions), David Barton (solving algebraic systems of equations), Richard Bogen (special functions), Bill Dubuque (indefinite integration, limits, power series, number theory, special functions, functional equations, pattern matching, sign queries, Gröbner, TriangSys), Richard Fateman (rational functions, pattern matching, arbitrary precision floating-point), Michael Genesereth (comparison, knowledge database), Jeff Golden (simplifier, language, system), R. W. Gosper (definite summation, special functions, simplification, number theory), Carl Hoffman (general simplifier, macros, non-commutative simplifier, ports to Multics and LispM, system, visual equation editor), Charles Karney (plotting), John Kulp, Ed Lafferty (ODE solution, special functions), Stavros Macrakis (real/imaginary parts, compiler, system), Richard Pavelle (indicial tensor calculus, general relativity package, ordinary and partial differential equations), David A. Spear (Gröbner),^[6][7] Barry Trager (algebraic integration,^[8] factoring, Gröbner^[9]), Paul S. Wang (polynomial factorization and GCD, complex numbers, limits, definite integration, Fortran and LaTeX code generation), David Y. Y. Yun (polynomial GCDs), Gail Zacharias (Gröbner)^[9][10] and Rich Zippel (power series, polynomial factorization, number theory, combinatorics).

Macsyma was written in Maclisp, and was, in some cases, a key motivator for improving that dialect of Lisp in the areas of numerical computing, efficient compilation and language design. Maclisp itself ran primarily on PDP-6 and PDP-10 computers, but also on the Multics OS and on the Lisp Machine architectures. Macsyma was one of the largest, if not the largest, Lisp programs of the time.

In response to a request from Richard Fateman, then a professor at UC Berkeley, MIT provided a temporary license for Macsyma code in 1979. This was almost immediately brought up on the new VAX-11/780 computer using a dialect of Maclisp, Franz Lisp, written by Fateman's group. MIT reluctantly licensed versions of VAX Macsyma from Berkeley to about 50 universities, starting with Caltech, under condition that the licenses would be revoked when negotiations for a proper license were complete. Indeed, this happened when the negotiation with Symbolics (see below) was complete. Symbolics was reluctant to license the VAX product, since VAX constituted competitive hardware to their own Lisp Machines and therefore suppressed the VAX software for five years. UC Berkeley also brought up copies of Macsyma on Motorola 68000-based systems, most notably Sun workstations. Symbolics suppressed those as well. At the same time Fateman worked to change the (now revoked) temporary license for Macsyma into something more permanent.

Ultimately, the United States Department of Energy imposed on MIT the requirement that they release a copy into the National Energy Software Center (NESC) library in 1982, while allowing MIT to assert restrictions in terms of high price and no redistribution. This was intended to protect the technology transfer to Symbolics. (Such restrictions have been since lifted, 2002 or so.) This so-called DOE Macsyma had been rewritten at MIT in a pre-Common Lisp dialect called NIL lisp, incompletely implemented on VAX/VMS, an unpopular system in academia, where Berkeley VAX Unix was common. DOE Macsyma formed the basis for the much-later open source Maxima system.

In 1981, Moses and Richard Pavelle, an MIT staff member and proponent of applying Macsyma to engineering and science, attempted to form a company to commercialize Macsyma. Pavelle had written many scientific papers using Macsyma. With the papers in hand, Pavelle and Moses approached several venture capital firms that showed interest in funding a Macsyma startup. With potential deals close to finalization, MIT suddenly decided that MIT personnel should not profit directly from MIT developments. In early 1982, Macsyma was licensed by MIT to Arthur D. Little, Inc. (ADL), which became the broker for Macsyma and soon licensed Macsyma to Symbolics in late 1982. Moses was forced out of the picture by ADL and Pavelle became the head of the Macsyma division at Symbolics. It was evident that Symbolics was not so much interested in the code as in keeping Macsyma out of the software catalog of its competitor in the Lisp Machine business, LMI. The business arrangement between Symbolics and Arthur D. Little required a royalty payment to ADL of 15% of Macsyma gross sales. This royalty led to speculation on the desire of MIT and ADL to see MACSYMA prosper. The development of Macsyma continued at Symbolics despite the fact that it was seen as a diversion from the sales of Lisp machines, which Symbolics considered to be their main business despite the fact that Macsyma sales and the leveraged sales of Lisp Machines reached 10% of overall sales at Symbolics within two years. Despite resistance from many in Symbolics, Macsyma was released for DEC VAX computers and Sun workstations using Berkeley's Franz Lisp in the early to mid 80s.

However, problems at Symbolics over the sales of Macsyma on computers other than Symbolics' eventually led to the decline of Macsyma sales. In the first half of 1986, Macsyma revenues were lower than in the first half of 1985, in a growing industry. Wolfram's SMP program and Waterloo Maple were growing at this time, although MACSYMA was easily superior to these other packages in symbolic math.

Pavelle ran the Macsyma division at Symbolics until early 1986. In the second half of 1986 Richard Petti became the manager of the Macsyma business to reduce the sales and avoid conflict with employees in Symbolics. Macsyma cut headcount but expanded its sales force and marketing, and focused its developers more on features that customers asked for. (For example, the Spear-Zacharias (Gröbner) basis algorithm implemented in the 1970s at MIT was not installed in a shipping Macsyma until 1987.) In 1987, Macsyma annual revenues roughly doubled. MACSYMA became more user friendly: documentation and on-line help were reorganized and expanded; some command names were changed to be more mnemonic. The system was ported to Common Lisp (by Kent M. Pitman). Petti argued to Symbolics management that Macsyma was a 'strategic business unit' that should be funded based on its performance and potential, not based on the prospects of the main workstation business. However, Symbolics cut the Macsyma headcount after this period of sharp growth; Symbolics tried to use Macsyma as a tiny cash cow to fund the losses of the workstation business that was 30 times its size.

The greatest product weakness of Macsyma was its numerical analysis. Mastering symbolic math was a herculean task; but numerical capabilities were critical to get a piece of the much larger engineering and lower-end scientific markets. At MIT Macsyma had a link to the IMSL (now a product of Perforce Software) numerical libraries, but this link was severed when Macsyma moved to Symbolics. Lisp developers at Symbolics generally believed that numerical analysis was an old technology that was not important for Lisp applications, so they declined to invest in it. Double-precision arithmetic in Macsyma (on a PC version) was about six times slower than Fortran. Also Macsyma's matrices were implemented as lists of lists, which cost another factor of perhaps 5–10 in speed for key algorithms. Macsyma did not have many of the basic algorithms of numerical linear algebra, such as LU decomposition.

In 1987-88, the Macsyma group tried to build a PC Macsyma with Gold Hill Lisp. (Earlier, Symbolics had killed its own project to build a Lisp compiler for standard computers to avoid competing with Lisp machine sales. This was a controversial move that, by some accounts, was undertaken without approval of senior management. Symbolics also declined to cooperate with Sun to make Lisp available on Sun workstations for the same reason.) Gold Hill Lisp was too unstable, and its weak architecture made it impossible for Gold Hill to eliminate the bugs. This was a crucial failure for Macsyma. It meant that Macsyma could not respond on PCs when Mathematica appeared on Apple computers in mid-1988. Macsyma appeared on Windows PCs in August 1989 using the CLOE Lisp from Symbolics. However, the Macsyma staff was too small and lacked the mix of skills needed to add the kind of graphics, notebook interface and better numerics that Mathematica had.

By 1989, it was clear to Petti that Symbolics would implode due to poor product strategy, and that it would take Macsyma with it. (Some product issues: (1) Symbolics' software was designed for MIT-class software developers without enough concessions to others. (2) Symbolics' world-class software was losing share in the market due to dependence on very high-cost hardware. (3) In moving to VLSI hardware in the mid-1980s, Symbolics converted from 36-bit word size to a 40-bit, without justification from the market for the enormous development cost of this change.) Macsyma could not assemble a buy-out team due to lack of cooperation from MIT. After keeping reasonably quiet since 1986 about the product issues, Petti tried in late 1988 to persuade Symbolics to adopt a software-only or board-level strategy; but the fourth president in four years would hear nothing of it. In 1990 Petti left Symbolics for a start-up.

Macsyma, Inc., was founded in 1992 by Russell Noftsker (chairman, who had co-founded Symbolics) and Richard Petti (president, who had turned around Macsyma for a time at Symbolics) by raising funds and purchasing the rights to Macsyma from the ailing Symbolics.^[11] Although the market was growing fast, Macsyma sales in 1991 and early 1992 were still falling rapidly. Macsyma's market share in symbolic math software had fallen from 70% in 1987 to 1% in 1992. By 1993, market growth had slowed and the market had standardized on Mathematica and Maple. The competitors had development staffs that were 4-8 times as large as that of Macsyma Inc. throughout the 1990s.

In early 1995 the company shipped Macsyma 2.0.5, with many improvements: On Wester's large test of symbolic math, Macsyma 2.0.5 scored 10% better than Maple and 15% better than Mathematica.^[12] Although Macsyma 2.0.5 was still very slow at numerics, it had a greatly strengthened portfolio of numerical analysis and linear algebra routines. (In 1996 Macsyma added LAPACK which greatly increased the speed of most numerical linear algebra.) .

The development team, at this time, included Jeff Golden (language, compiler, etc.), Bill Gosper (special functions, summation), Howard Cannon (user interface, optimization), and several consultants. Bill Dubuque worked on integration, equation solving, database, and optimization. Christopher Stacy and David H. Wood added MATLAB's language and matrix functionality. Other developers made major contributions in numerical analysis, graphics, and help systems.

The company could not continue indefinitely to outperform staffs 4-8 times as large and reverse the market momentum. Market share did not increase above 2%, because the competitors were entrenched in all key accounts, converting to a new product is very costly in learning time, and market growth declined. Also, starting in 1992 or 1993, Mathsoft engaged in a strategy of spending $10 million on direct mail at very low prices, which won much of the remaining growth in the symbolic math market, just when Macsyma Inc. was struggling to rebuild its world-class product.

In 1999, Macsyma were acquired by Tenedos LLC, a holding company that previously had purchased Symbolics. The holding company has not re-released or resold Macsyma, but Macsyma continues to be distributed by Symbolics for Microsoft Windows versions up to Windows XP.^[13]

There is also a GPL-licensed version, called Maxima, based on the 1982 version of the DOE Macsyma, subsequently adapted for Common Lisp and enhanced by William Schelter. It is under active development, and can be compiled under several Common Lisp systems. Downloadable executables for Linux, Microsoft Windows, Mac OS X and other systems, including graphical user interfaces are available. Maxima does not include many of the numerous features added during the period of commercial development between 1982–1999, but is a current, free, open codebase that includes numerous additional features, several alternative front ends, and works with a number of Common Lisp engines. While this has resulted in some incompatibilities between Macsyma and Maxima, programs written in the Macsyma algebraic language can often be run, with only minor changes, in either system.

ITS Macsyma from around 1982 is available, with most files in a directory called MAXSRC.

• Richard Petti's summary of the history of commercial Macsyma 30 October 2003
• The Macsyma Saga Richard Petti, 2 November 2003
• Symbolics Macsyma