Abstract
We report here on an experimental investigation of LTL satisfiability checking via a reduction to model checking. By using large LTL formulas, we offer challenging model-checking benchmarks to both explicit and symbolic model checkers. For symbolic model checking, we use CadenceSMV, NuSMV, and SAL-SMC. For explicit model checking, we use SPIN as the search engine, and we test essentially all publicly available LTL translation tools. Our experiments result in two major findings. First, most LTL translation tools are research prototypes and cannot be considered industrial quality tools. Second, when it comes to LTL satisfiability checking, the symbolic approach is clearly superior to the explicit approach.
Similar content being viewed by others
References
Ammons, G., Mandelin, D., Bodik, R., Larus, J.R.: Debugging temporal specifications with concept analysis. In: Proceedings of the ACM Conference on PLDI, pp. 182–195 (2003)
Armoni, R., Fix, L., Flaisher, A., Grumberg, O., Piterman, N., Tiemeyer, A., Vardi, M.Y.: Enhanced vacuity detection for linear temporal logic. In: Proceedings of the 15th International Conference on CAV. Springer, Berlin (2003)
Beer I., Ben-David S., Eisner C., Rodeh Y.: Efficient detection of vacuity in ACTL formulas. Formal Methods Syst. Des. 18(2), 141–162 (2001)
Bensalem S., Ganesh V., Lakhnech Y., Muñoz C., Owre S., Rueß H., Rushby J., Rusu V., Saïdi H., Shankar N., Singerman E., Tiwari A.: An overview of SAL. In: Michael Holloway, C. (eds) LFM 2000: Fifth NASA Langley Formal Methods Workshop, pp. 187–196. NASA Langley Research Center, Hampton, VA (2000)
Bloem, R., Ravi, K., Somenzi, F. (1999) Efficient decision procedures for model checking of linear time logic properties. In: Proceedings of the 11th International Conference on CAV. Lecture Notes in Computer Science, vol. 1633, pp. 222–235. Springer, Berlin (1999)
Brayton, R.K., Hachtel, G.D., Sangiovanni-Vincentelli, A. Somenzi, F., Aziz, A., Cheng, S.-T., Edwards, S., Khatri, S., Kukimoto, T., Pardo, A., Qadeer, S., Ranjan, R.K., Sarwary, S., Shiple, T.R., Swamy, G., Villa, T.: VIS: a system for verification and synthesis. In: Proceedings of the 8th International Conference on CAV. Lecture Notes in Computer Science, vol. 1102, pp. 428–432. Springer, Berlin (1996)
Bryant R.E.: Graph-based algorithms for boolean-function manipulation. IEEE Trans. Comput. C-35(8), 677–691 (1986)
Burch J.R., Clarke E.M., McMillan K.L., Dill D.L., Hwang L.J.: Symbolic model checking: 1020 states and beyond. Inf. Comput. 98(2), 142–170 (1992)
Bustan, D., Flaisher, A., Grumberg, O., Kupferman, O., Vardi, M.Y.: Regular vacuity. In: CHARME. LNCS, vol. 3725, pp. 191–206. Springer, Berlin (2005)
Cimatti A., Clarke E.M., Giunchiglia F., Roveri M.: NuSMV: a new symbolic model checker. Int. J. Softw. Tools Technol. Transf. 2(4), 410–425 (2000)
Clarke E.M., Grumberg O., Hamaguchi K.: Another look at LTL model checking. Formal Methods Syst. Des. 10(1), 47–71 (1997)
Clarke E.M, Grumberg O., Peled D.: Model Checking. MIT Press, Cambridge (1999)
Courcoubetis C., Vardi M.Y., Wolper P., Yannakakis M.: Memory efficient algorithms for the verification of temporal properties. Formal Methods Syst. Des. 1, 275–288 (1992)
Couvreur, J.-M.: On-the-fly verification of linear temporal logic. In: Proceedings of FM, pp. 253–271 (1999)
Daniele, N., Guinchiglia, F., Vardi, M.Y.: Improved automata generation for linear temporal logic. In: Proceedigs of the 11th International Conference on CAV. LNCS, vol. 1633, pp. 249–260. Springer, Berlin (1999)
de Moura, L., Owre, S., Rueß, H., Rushby, J., Shankar, N., Sorea, M., Tiwari, A.: SAL 2. In: Alur, R., Peled, D. (eds.) Computer-Aided Verification, CAV 2004. Lecture Notes in Computer Science, vol. 3114, pp. 496–500. Springer, Boston (2004)
Duret-Lutz, A., Poitrenaud, D.: SPOT: An extensible model checking library using transition-based generalized büchi automata. In: Proceedings of the 12th International Workshop on MASCOTS, pp. 76–83. IEEE Computer Society, USA (2004)
Emerson E.A.: Temporal and modal logic. In: Van Leeuwen, J. (eds) Handbook of Theoretical Computer Science, vol. B, ch. 16, pp. 997–1072. Elsevier MIT Press, Amsterdam (1990)
Emerson, E.A., Lei, C.L.: Efficient model checking in fragments of the propositional μ-calculus. In: LICS, 1st Symp. pp. 267–278, Cambridge (1986)
Etessami, K., Holzmann, G.J.: Optimizing Büchi automata. In: Proceedings of the 11th International Conference on CONCUR. Lecture Notes in CS 1877, pp. 153–167. Springer, Berlin (2000)
Fritz, C.: Constructing Büchi automata from linear temporal logic using simulation relations for alternating büchi automata. In: Proceedings of the 8th International conference on CIAA. Lecture Notes in Computer Science, vol. 2759, pp. 35–48. Springer, Berlin (2003)
Fritz, C.: Concepts of automata construction from LTL. In: Proceedings of the 12th International Conference on LPAR. Lecture Notes in Computer Science, vol. 3835, pp. 728–742. Springer, Berlin (2005)
Gastin, P., Oddoux, D.: Fast LTL to Büchi automata translation. In: Proceedings of the 13th International Conference on CAV. LNCS, vol. 2102, pp. 53–65. Springer, Berlin (2001)
Geldenhuys, J., Hansen, H.: Larger automata and less work for LTL model checking. In: Model Checking Software, 13th Int’l SPIN Workshop. LNCS, vol. 3925, pp. 53–70. Springer, Berlin (2006)
Geldenhuys, J., Valmari, A.: Tarjan’s algorithm makes on-the-fly LTL verification more efficient. In: Proceedings of the 10th International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, vol. 2988, pp. 205–219. Springer, Berlin (2004)
Gerth R., Peled D., Vardi M.Y., Wolper P.: Simple on-the-fly automatic verification of linear temporal logic. In: Dembiski, P., Sredniawa, M. (eds) Protocol Specification, Testing, and Verification, pp. 3–18. Chapman & Hall, London (1995)
Giannakopoulou, D., Lerda, F.: From states to transitions: Improving translation of LTL formulae to Büchi automata. In: Proceedings of 22 IFIP International Conference on FORTE (2002)
Gurfinkel, A., Chechik, M.: Extending extended vacuity. In: 5th International Conferene on FMCAD. Lecture Notes in Computer Science, vol. 3312, pp 306–321. Springer, Berlin (2004)
Gurfinkel, A., Chechik, M.: How vacuous is vacuous. In: 10th International Conference on TACAS. Lecture Notes in Computer Science, vol. 2988, pp. 451–466. Springer, Berlin (2004)
Holzmann G.J.: The model checker SPIN. IEEE Trans. Softw. Eng. 23(5), 279–295 (1997) Special issue on Formal Methods in Software Practice
Kupferman, O.: Sanity checks in formal verification. In: Proceedings of the 17th International Conference on CONCUR. Lecture Notes in Computer Science, vol. 4137, pp. 37–51. Springer, Berlin (2006)
Kupferman O., Vardi M.Y.: Vacuity detection in temporal model checking. J. Softw. Tools Technol. Transf. 4(2), 224–233 (2003)
Kurshan R.P.: FormalCheck User’s Manual. Cadence Design, Inc., San Jose (1998)
McMillan, K.: The SMV language. Technical report, Cadence Berkeley Lab (1999)
McMillan K.L.: Symbolic Model Checking. Kluwer Academic Publishers, Dordrecht (1993)
Namjoshi, K.S.: An efficiently checkable, proof-based formulation of vacuity in model checking. In: 16th CAV. LNCS, vol. 3114, pp. 57–69. Springer, Berlin (2004)
Pan, G., Sattler, U.,Vardi, M.Y.: BDD-based decision procedures for K. In: Proceedings of the 18th International conference on CADE. LNCS, vol. 2392, pp. 16–30. Springer, Berlin (2002)
Piterman N., Vardi M.Y.: From bidirectionality to alternation. Theor. Comput. Sci. 295(1–3), 295–321 (2003)
Purandare, M., Somenzi, F.: Vacuum cleaning CTL formulae. In: Proceeding of the 14th Conference on CAV. Lecture Notes in Computer Science, pp. 485–499. Springer, Berlin (2002)
Sebastiani, R., Tonetta, S.: “more deterministic” vs. “smaller” büchi automata for efficient LTL model checking. In: CHARME, pp. 126–140. Springer, Berlin (2003)
Sebastiani, R., Tonetta, S., Vardi, M.Y.: Symbolic systems, explicit properties: on hybrid approaches for LTL symbolic model checking. In: Proceedings of the 17th International Conference on CAV. Lecture Notes in Computer Science, vol. 3576, pp. 350–373. Springer, Berlin (2005)
Somenzi, F., Bloem, R.: Efficient Büchi automata from LTL formulae. In: Proceedings of the 12th International Conference on CAV. LNCS, vol. 1855, pp. 248–263. Springer, Berlin (2000)
Tauriainen H., Heljanko K.: Testing LTL formula translation into Büchi automata. STTT Int. J. Softw. Tools Technol. Transf. 4(1), 57–70 (2002)
Thirioux X.: Simple and efficient translation from LTL formulas to Büchi automata. Electr. Notes Theor. Comput. Sci. 66(2), 145–159 (2002)
Vardi M.Y.: Nontraditional applications of automata theory. In: Proceedings of the International conference on STACS. LNCS, vol. 789, pp. 575–597. Springer, Berlin (1994)
Vardi M.Y.: Automata-theoretic model checking revisited. In: Proceedings of the 7th International Conference on Verification, Model Checking, and Abstract Interpretation. LNCS, vol. 4349, pp. 137–150. Springer, Berlin (2007)
Vardi M.Y., Wolper, P.: An automata-theoretic approach to automatic program verification. In: Proceeding of the 1st LICS, pp. 332–344, Cambridge (1986)
Vardi M.Y., Wolper P.: Reasoning about infinite computations. Inf. Comput. 115(1), 1–37 (1994)
Author information
Authors and Affiliations
Corresponding author
Additional information
An earlier version of this paper appeared in SPIN’07.
Work contributing to this paper was completed at Rice University, Cambridge University, and NASA, and was supported in part by the Rice Computational Research Cluster (Ada), funded by NSF under Grant CNS-0421109 and a partnership between Rice University, AMD and Cray.
The use of names of software tools in this report is for accurate reporting and does not constitute an official endorsement, either expressed or implied, of such software by the National Aeronautics and Space Administration.
Rights and permissions
About this article
Cite this article
Rozier, K.Y., Vardi, M.Y. LTL satisfiability checking. Int J Softw Tools Technol Transfer 12, 123–137 (2010). https://doi.org/10.1007/s10009-010-0140-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10009-010-0140-3