More Web Proxy on the site http://driver.im/

research-article

Managing data constraints in database-backed web applications

Authors:

Shan LuAuthors Info & Claims

ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

Pages 1098 - 1109

https://doi.org/10.1145/3377811.3380375

Published: 01 October 2020 Publication History

Abstract

Database-backed web applications manipulate large amounts of persistent data, and such applications often contain constraints that restrict data length, data value, and other data properties. Such constraints are critical in ensuring the reliability and usability of these applications. In this paper, we present a comprehensive study on where data constraints are expressed, what they are about, how often they evolve, and how their violations are handled. The results show that developers struggle with maintaining consistent data constraints and checking them across different components and versions of their web applications, leading to various problems. Guided by our study, we developed checking tools and API enhancements that can automatically detect such problems and improve the quality of such applications.

References

[1]

Diaspora-5090. https://github.com/diaspora/diaspora/issues/5090.

[2]

Discourse. A blog application. https://github.com/discourse/discourse/.

[3]

Discourse-89148. https://meta.discourse.org/t/89148.

[4]

Discourse Import Scripts. A blog application. https://github.com/discourse/discourse/tree/master/script/import_scripts.

[5]

Django-cms. An enterprise content management system. https://github.com/divio/django-cms/.

[6]

Django Validator Function. https://docs.djangoproject.com/en/2.2/ref/validators/.

[7]

Gitlab-24493. https://gitlab.com/gitlab-org/gitlab-ce/issues/24493.

[8]

Gitlab-36919. https://gitlab.com/gitlab-org/gitlab-ce/issues/36919.

[9]

Gitlab database migrate. https://github.com/gitlabhq/gitlabhq/tree/master/db/migrate.

[10]

Gitlab releases. https://about.gitlab.com/releases/.

[11]

Hibernate Validator Annotation. https://hibernate.org/validator/documentation/getting-started/.

[12]

Hyperloop. https://hyperloop-rails.github.io/vibranium/.

[13]

Rails Uniqueness API. https://github.com/rails/rails/blob/master/activerecord/lib/active_record/validations/uniqueness.rb#L165/.

[14]

Rails Uniqueness Problem. https://thoughtbot.com/blog/the-perils-of-uniqueness-validations.

[15]

Redash. An application to connect your company's data. https://github.com/getredash/redash/.

[16]

redmine-24283. https://www.redmine.org/issues/24283.

[17]

Redmine-25235. http://www.redmine.org/issues/25235/.

[18]

Redmine-9394. http://www.redmine.org/issues/9394/.

[19]

redmine, a project management application. https://redmine.org/. (????).

[20]

Spree. A ecommerce application. https://github.com/spree/spree/.

[21]

Spree-3829. https://github.com/spree/spree/issues/3829.

[22]

Spree-4123. https://github.com/diaspora/diaspora/issues/4123.

[23]

Spree-6673. https://github.com/spree/spree/issues/6673.

[24]

Zulip. A powerful team chat system. https://github.com/zulip/zulip/.

[25]

Muath Alkhalaf, Shauvik Roy Choudhary, Mattia Fazzini, Tevfik Bultan, Alessandro Orso, and Christopher Kruegel. 2012. Viewpoints: differential string analysis for discovering client-and server-side input validation inconsistencies. In Proceedings of the 2012 International Symposium on Software Testing and Analysis. ACM, 56--66.

Digital Library

[26]

Ivan Bocić, Tevfik Bultan, and Nicolás Rosner. 2019. Inductive verification of data model invariants in web applications using first-order logic. Automated Software Engineering 26, 2 (2019), 379--416.

Digital Library

[27]

Loredana Caruccio, Giuseppe Polese, and Genoveffa Tortora. 2016. Synchronization of queries and views upon schema evolutions: A survey. ACM Transactions on Database Systems (TODS) 41, 2 (2016), 9.

Digital Library

[28]

Tse-Hsun Chen, Weiyi Shang, Zhen Ming Jiang, Ahmed E. Hassan, Mohamed Nasser, and Parminder Flora. 2014. Detecting Performance Anti-patterns for Applications Developed Using Object-relational Mapping. In ICSE. 1001--1012.

[29]

Tse-Hsun Chen, Weiyi Shang, Zhen Ming Jiang, Ahmed E. Hassan, Mohamed Nasser, and Parminder Flora. 2016. Finding and evaluating the performance impact of redundant data access for applications that are developed using object-relational mapping frameworks. In ICSE. 1148--1161.

[30]

Shumo Chu, Chenglong Wang, Konstantin Weitz, and Alvin Cheung. 2017. Cosette: An Automated Prover for SQL. In CIDR.

[31]

Shumo Chu, Konstantin Weitz, Alvin Cheung, and Dan Suciu. 2017. HoTTSQL: Proving Query Rewrites with Univalent SQL Semantics. In PLDI. 510--524.

[32]

Alin Deutsch, Monica Marcus, Liying Sui, Victor Vianu, and Dayou Zhou. 2005. A verifier for interactive, data-driven web applications. In Proceedings of the 2005 ACM SIGMOD international conference on Management of data. ACM, 539--550.

Digital Library

[33]

Nikolaos Karapanos, Alexandros Filios, Raluca Ada Popa, and Srdjan Capkun. 2016. Verena: End-to-end integrity protection for web applications. In 2016 IEEE Symposium on Security and Privacy (SP). IEEE, 895--913.

[34]

Feifei Li, Bin Wu, Ke Yi, and Zhuoyue Zhao. 2016. Wander join: Online aggregation via random walks. In Proceedings of the 2016 International Conference on Management of Data. ACM, 615--629.

Digital Library

[35]

Guy Lohman. Is Query Optimization a "Solved" Problem? https://wp.sigmod.org/?p=1075.

[36]

Joseph P Near and Daniel Jackson. 2014. Derailer: interactive security analysis for web applications. In Proceedings of the 29th ACM/IEEE international conference on Automated software engineering. ACM, 587--598.

Digital Library

[37]

Kai Pan, Xintao Wu, and Tao Xie. 2014. Guided test generation for database applications via synthesized database interactions. ACM Transactions on Software Engineering and Methodology (TOSEM) 23, 2 (2014), 12.

Digital Library

[38]

Marija Selakovic and Michael Pradel. 2016. Performance issues and optimizations in javascript: an empirical study. In ICSE. 61--72.

[39]

Rohit Singh, Vamsi Meduri, Ahmed Elmagarmid, Samuel Madden, Paolo Papotti, Jorge-Arnulfo Quiané-Ruiz, Armando Solar-Lezama, and Nan Tang. 2017. Generating concise entity matching rules. In Proceedings of the 2017 ACM International Conference on Management of Data. ACM, 1635--1638.

Digital Library

[40]

Chenglong Wang, Alvin Cheung, and Rastislav Bodík. 2018. Speeding up symbolic reasoning for relational queries. PACMPL 2, OOPSLA (2018), 157:1--157:25.

[41]

Yuepeng Wang, Isil Dillig, Shuvendu K. Lahiri, and William R. Cook. 2017. Verifying Equivalence of Database-driven Applications. In Proceedings of the ACM on Programming Languages. 56:1--56:29.

[42]

Yuepeng Wang, James Dong, Rushi Shah, and Isil Dillig. 2019. Synthesizing database programs for schema refactoring. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 286--300.

Digital Library

[43]

Cong Yan and Alvin Cheung. Leveraging Lock Contention to Improve OLTP Application Performance. Proc. VLDB Endow. (2016), 444--455.

Digital Library

[44]

Cong Yan and Alvin Cheung. 2019. Generating Application-Specific Data Layouts for In-memory Databases. Proc. VLDB Endow. (2019), 1513--1525.

Digital Library

[45]

Cong Yan, Junwen Yang, Alvin Cheung, and Shan Lu. 2017. Understanding Database Performance Inefficiencies in Real-world Web Applications. In CIKM.

[46]

Jean Yang, Travis Hance, Thomas H Austin, Armando Solar-Lezama, Cormac Flanagan, and Stephen Chong. 2016. Precise, dynamic information flow for database-backed applications. ACM SIGPLAN Notices 51, 6 (2016), 631--647.

Digital Library

[47]

Junwen Yang, Pranav Subramaniam, Shan Lu, Cong Yan, and Alvin Cheung. 2018. How not to structure your database-backed web applications: a study of performance bugs in the wild. In ICSE. 800--810.

[48]

Junwen Yang, Pranav Subramaniam, Shan Lu, Cong Yan, and Alvin Cheung. 2018. PowerStation: Automatically detecting and fixing inefficiencies of database-backed web applications in IDE. In FSE. 884--887.

Cited By

Huang YShi CLu JLi HMeng HLi LLuo BLiao XXu JKirda ELie D(2024)Detecting Broken Object-Level Authorization Vulnerabilities in Database-Backed ApplicationsProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690227(2934-2948)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3690227
Belgacem HLi XBianculli DBriand L(2024)Learning-based Relaxation of Completeness Requirements for Data Entry FormsACM Transactions on Software Engineering and Methodology10.1145/363570833:3(1-32)Online publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1145/3635708
Wang TZhi CZhou XWu JYin JDeng SJust RFraser G(2023)Data Constraint Mining for Automatic Reconciliation Scripts GenerationProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598122(1119-1130)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598122
Show More Cited By

Recommendations

Protecting Data Integrity of Web Applications with Database Constraints Inferred from Application Code
ASPLOS 2023: Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2

Database-backed web applications persist a large amount of production data and have high requirements for integrity. To protect data integrity against application code bugs and operator mistakes, most RDBMSes allow application developers to specify ...
Managing data constraints in database-backed web applications
ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

There are often constraints associated with data used in software, describing the expected length, value, uniqueness, and other properties of the stored data. Correctly specifying and checking such constraints are crucial for reliability, ...
Leveraging Application Data Constraints to Optimize Database-Backed Web Applications

Exploiting the relationships among data is a classical query optimization technique. As persistent data is increasingly being created and maintained programmatically, prior work that infers data relationships from data statistics misses an important ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

June 2020

1640 pages

ISBN:9781450371216

DOI:10.1145/3377811

General Chairs:
Gregg Rothermel
North Carolina State University
,
Doo-Hwan Bae
KAIST, South Korea

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

KIISE: Korean Institute of Information Scientists and Engineers
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Qualifiers

Research-article

Funding Sources

NSF

Conference

ICSE '20

Sponsor:

SIGSOFT

ICSE '20: 42nd International Conference on Software Engineering

June 27 - July 19, 2020

Seoul, South Korea

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
416
Total Downloads

Downloads (Last 12 months)98
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Huang YShi CLu JLi HMeng HLi LLuo BLiao XXu JKirda ELie D(2024)Detecting Broken Object-Level Authorization Vulnerabilities in Database-Backed ApplicationsProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3690227(2934-2948)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3690227
Belgacem HLi XBianculli DBriand L(2024)Learning-based Relaxation of Completeness Requirements for Data Entry FormsACM Transactions on Software Engineering and Methodology10.1145/363570833:3(1-32)Online publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1145/3635708
Wang TZhi CZhou XWu JYin JDeng SJust RFraser G(2023)Data Constraint Mining for Automatic Reconciliation Scripts GenerationProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598122(1119-1130)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597926.3598122
Wang CFan GYao PPan FZhang CGrundy JPollock LPenta M(2023)Verifying Data Constraint Equivalence in FinTech SystemsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00117(1329-1341)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00117
Tang WWang CYao PWu RFu XFan GZhang CBissyandé TKlein JBird CSarro F(2023)DCLink: Bridging Data Constraint Changes and Implementations in FinTech SystemsProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00170(914-925)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1109/ASE56229.2023.00170
Florez JPerry JWei SMarcus ADwyer MDamian DZeller A(2022)Retrieving data constraint implementations using fine-grained code patternsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510167(1893-1905)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510167
Florez JMoreno LZhang ZWei SMarcus A(2022)An empirical study of data constraint implementations in JavaEmpirical Software Engineering10.1007/s10664-022-10175-w27:5Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10664-022-10175-w

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten