research-article

Parallel virtual machines with RPython

Authors:

Remigius Meier,

Armin Rigo,

Thomas R. GrossAuthors Info & Claims

DLS 2016: Proceedings of the 12th Symposium on Dynamic Languages

Pages 48 - 59

https://doi.org/10.1145/2989225.2989233

Published: 01 November 2016 Publication History

Get Access

Abstract

The RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support parallel execution since the framework relies on a Global Interpreter Lock (GIL): PyPy serialises the execution of multi-threaded Python programs.

We describe the rationale and design of a new parallel execution model for RPython that allows the generation of parallel virtual machines while leaving the language semantics unchanged. This model then allows different implementations of concurrency control, and we discuss an implementation based on a GIL and an implementation based on Software Transactional Memory (STM).

To evaluate the benefits of either choice, we adapt PyPy to work with both implementations (GIL and STM). The evaluation shows that PyPy with STM improves the runtime of a set of multi-threaded Python programs over PyPy with a GIL by factors in the range of 1.87x up to 5.96x when executing on a processor with 8 cores.

References

[1]

“The IronPython Project,” http://ironpython.net/.

Google Scholar

[2]

“The Jython Project,” http://www.jython.org/.

Google Scholar

[3]

Nicholas Riley and Craig Zilles, “Hardware Transactional Memory Support for Lightweight Dynamic Language Evolution,” OOPSLA ’06, pp. 998–1008, ACM.

Digital Library

Google Scholar

[4]

Rei Odaira, Jose G. Castanos, and Hisanobu Tomari, “Eliminating Global Interpreter Locks in Ruby Through Hardware Transactional Memory,” PPoPP ’14, pp. 131–142, ACM.

Digital Library

Google Scholar

[5]

Fuad Tabba, “Adding Concurrency in Python Using a Commercial Processor’s Hardware Transactional Memory Support,” SIGARCH Comput. Archit. News, vol. 38, no. 5, pp. 12–19, Apr. 2010.

Digital Library

Google Scholar

[6]

Yu David Liu, Xiaoqi Lu, and Scott F. Smith, “Coqa: Concurrent Objects with Quantized Atomicity,” CC’08/ETAPS’08, pp. 260–275, Springer-Verlag.

Digital Library

Google Scholar

[7]

Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski, and Armin Rigo, “Tracing the Meta-level: PyPy’s Tracing JIT Compiler,” ICOOOLPS ’09, pp. 18–25, ACM.

Digital Library

Google Scholar

[8]

“CPython,” https://www.python.org/.

Google Scholar

[9]

“The JRuby Project,” http://jruby.org/.

Google Scholar

[10]

Colin Blundell, E. Lewis, and Milo Martin, “Unrestricted Transactional Memory: Supporting I/O and System Calls Within Transactions,” Technical Reports (CIS), May 2006.

Google Scholar

[11]

“PyPy Source on Bitbucket,” https://bitbucket.org/ pypy/pypy.

Google Scholar

[12]

Remigius Meier, Armin Rigo, and Thomas Gross, “A Transactional Memory System for Parallel Python,” https://bitbucket.org/pypy/extradoc/raw/ 12bda771698925b8296808959e7b830aef8b78b8/ talk/dls2014/paper/paper.pdf, Aug. 2014.

Google Scholar

[13]

“PyPy Benchmarks on Bitbucket,” https://bitbucket. org/pypy/benchmarks.

Google Scholar

[14]

Christian Wimmer and Thomas Würthinger, “Truffle: A Selfoptimizing Runtime System,” SPLASH ’12, pp. 13–14, ACM.

Digital Library

Google Scholar

Cited By

View all

Sela GPetrank E(2022)Concurrent sizeProceedings of the ACM on Programming Languages10.1145/35633006:OOPSLA2(345-372)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3563300
Meier RGross TMarr SFumero J(2019)Reflections on the compatibility, performance, and scalability of parallel PythonProceedings of the 15th ACM SIGPLAN International Symposium on Dynamic Languages10.1145/3359619.3359747(91-103)Online publication date: 20-Oct-2019
https://dl.acm.org/doi/10.1145/3359619.3359747
Faes MGross T(2018)Concurrency-aware object-oriented programming with rolesProceedings of the ACM on Programming Languages10.1145/32765002:OOPSLA(1-30)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276500
Show More Cited By

Index Terms

Parallel virtual machines with RPython
1. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

Reflections on the compatibility, performance, and scalability of parallel Python
DLS 2019: Proceedings of the 15th ACM SIGPLAN International Symposium on Dynamic Languages

Today's hardware is increasingly parallel, and to increase performance, applications must be able to use this parallelism. Hence, programming languages must provide the means for parallel execution. The language Python offers a multithreading, shared-...
Virtual machine design for parallel dynamic programming languages

To leverage the benefits of modern hardware, dynamic languages must support parallelism, and parallelism requires a virtual machine (VM) capable of parallel execution — a parallel VM. However, unrestricted concurrency and the dynamism of dynamic ...
Parallel virtual machines with RPython
DLS '16

The RPython framework takes an interpreter for a dynamic language as its input and produces a Virtual Machine (VM) for that language. RPython is being used to develop PyPy, a high-performance Python interpreter. However, the produced VM does not support ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

DLS 2016: Proceedings of the 12th Symposium on Dynamic Languages

November 2016

131 pages

ISBN:9781450344456

DOI:10.1145/2989225

Program Chair:
Roberto Ierusalimschy
PUC-Rio, Brazil

ACM SIGPLAN Notices Volume 52, Issue 2
DLS '16
February 2017
131 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3093334
Editor:
Matthew Fluet
Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

SIGAda: ACM Special Interest Group on Ada Programming Language

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLASH '16

Sponsor:

SIGPLAN

SPLASH '16: Conference on Systems, Programming, Languages, and Applications: Software for Humanity

November 1, 2016

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 32 of 77 submissions, 42%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
187
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Sela GPetrank E(2022)Concurrent sizeProceedings of the ACM on Programming Languages10.1145/35633006:OOPSLA2(345-372)Online publication date: 31-Oct-2022
https://dl.acm.org/doi/10.1145/3563300
Meier RGross TMarr SFumero J(2019)Reflections on the compatibility, performance, and scalability of parallel PythonProceedings of the 15th ACM SIGPLAN International Symposium on Dynamic Languages10.1145/3359619.3359747(91-103)Online publication date: 20-Oct-2019
https://dl.acm.org/doi/10.1145/3359619.3359747
Faes MGross T(2018)Concurrency-aware object-oriented programming with rolesProceedings of the ACM on Programming Languages10.1145/32765002:OOPSLA(1-30)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276500
Meier RRigo AGross T(2018)Virtual machine design for parallel dynamic programming languagesProceedings of the ACM on Programming Languages10.1145/32764792:OOPSLA(1-25)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276479
Daloze BTal AMarr SMössenböck HPetrank E(2018)Parallelization of dynamic languages: synchronizing built-in collectionsProceedings of the ACM on Programming Languages10.1145/32764782:OOPSLA(1-30)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276478
Lee KLee SLee J(2018)Interactive character animation by learning multi-objective controlACM Transactions on Graphics10.1145/3272127.327507137:6(1-10)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3272127.3275071
Li CPan HLiu YTong XSheffer AWang W(2018)Robust flow-guided neural prediction for sketch-based freeform surface modelingACM Transactions on Graphics10.1145/3272127.327505137:6(1-12)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3272127.3275051
Won JPark JLee J(2018)Aerobatics control of flying creatures via self-regulated learningACM Transactions on Graphics10.1145/3272127.327502337:6(1-10)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3272127.3275023
Hung SLai HYen DChen C(2017)Exploring the Effects of Team Collaborative Norms and Team Identification on the Quality of Individuals' Knowledge Contribution in TeamsACM SIGMIS Database: the DATABASE for Advances in Information Systems10.1145/3158421.315842848:4(80-106)Online publication date: 7-Nov-2017
https://dl.acm.org/doi/10.1145/3158421.3158428

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Reflections on the compatibility, performance, and scalability of parallel Python

Virtual machine design for parallel dynamic programming languages

Parallel virtual machines with RPython