More Web Proxy on the site http://driver.im/

research-article

Investigating open data portals automatically: a methodology and some illustrations

Authors:

Andreiwid Sheffer Correa,

Pär-Ola Zander,

Flavio Soares Correa da SilvaAuthors Info & Claims

dg.o '18: Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age

Article No.: 82, Pages 1 - 10

https://doi.org/10.1145/3209281.3209292

Published: 30 May 2018 Publication History

Abstract

Deploying a suitable open data platform is one of the most important requirements for succeeding in the provision of open data. Currently, there are several platforms available in the market ranging from the commercial ecosystem to free and open source software. However, we know less about the extent to which they are adopted and what they offer. This paper aims to provide a methodology to investigate this. The methodology is illustrated through studying adoption and use of open data software platforms through a comprehensive survey of 3,152 open data portals worldwide. We have identified 1,104 installations relying on the main existing platforms CKAN, Socrata, ArcGIS Open Data, and OpenDataSoft. To support our analysis, we have automatically fetched metadata about 1,921,636 stored datasets. Our findings indicated that there is a gap between the adoption and the effective use of open data platforms, particularly in terms of technology choice. These data are both from a descriptive and analytical point, non-trivial and showcase the relevance of the methodology. This work makes contributions regarding the development of methods to automatically survey open data platforms and provides insights about availability of open data portals based on the utilization of software platforms, organized by country and frequency of dataset updates.

References

[1]

Tim Berners-Lee. 2016. Five-star Open Data. (2016). Retrieved 2017-04-09 from http://5stardata.info/en/

[2]

Irina Bolychevsky. 2013. U.S. government's data portal Data.gov relaunched on CKAN | ckan - The open source data portal software. (May 2013). Retrieved 2014-12-02 from http://ckan.org/2013/05/23/data-gov-relaunch-on-ckan/

[3]

Ana Brandusescu, Carlos Iglesias, and Kristen Robinson. 2016. Open Data Barometer. Global Report. Fourth Edition. Technical Report. The World Wide Web Foundation. http://opendatabarometer.org/doc/4thEdition/ODB-4thEdition-GlobalReport.pdf

[4]

Katrin Braunschweig, Julian Eberius, Maik Thiele, and Wolfgang Lehner. 2012. The State of Open Data Limits of Current Open Data Platforms. In Proceedings of the 21st World Wide Web Conference 2012, Web Science Track at WWW'12, Lyon, France, April 16--20, 2012. ACM.

[5]

Kellyton dos Santos Brito, Marcos Antônio da Silva Costa, Vinicius Cardoso Garcia, and Silvio Romero de Lemos Meira. 2015. Is Brazilian Open Government Data Actually Open Data?: An Analysis of the Current Scenario. International Journal of E-Planning Research (IJEPR) 4, 2 (2015), 57--73.

Digital Library

[6]

Robyn Caplan, Timothy Davies, Asiya Wadud, Stefaan Verhulst, Jose Alonso, and Hania Farhan. 2014. Towards common methods for assessing open data: workshop report & draft framework. Technical Report. The World Wide Web Foundation, New York, USA. http://opendataresearch.org/sites/default/files/posts/Common%20Assessment%20Workshop%20Report.pdf

[7]

Peter Conradie and Sunil Choenni. 2014. On the barriers for local government releasing open data. Government Information Quarterly 31, Supplement 1 (June 2014), S10--S17.

[8]

Wikipedia contributors. 2018. International recognition of Kosovo --- Wikipedia, The Free Encyclopedia. (2018). https://en.wikipedia.org/w/index.php?title=International_recognition_of_Kosovo&oldid=834034249 {Online; accessed 8-April-2018}.

[9]

Andreiwid Sheffer Corrêa, Pedro Luiz Pizzigatti Corrêa, and Flávio Soares Corrêa da Silva. 2014. Transparency Portals Versus Open Government Data: An Assessment of Openness in Brazilian Municipalities. In Proceedings of the 15th Annual International Conference on Digital Government Research (dg.o '14). ACM, New York, NY, USA, 178--185.

Digital Library

[10]

Andreiwid Sheffer Correa, Evandro Couto de Paula, Pedro Luiz Pizzigatti Correa, and Flavio Soares CorrĂła da Silva. 2017. Transparency and open government data: A wide national assessment of data openness in Brazilian local governments. Transforming Government: People, Process and Policy 11, 1 (2017), 58--78.

[11]

Andreiwid Sheffer Corrêa and Pär-Ola Zander. 2017. Unleashing Tabular Content to Open Data: A Survey on PDF Table Extraction Methods and Tools. In Proceedings of the 18th Annual International Conference on Digital Government Research (dg.o '17). ACM, New York, NY, USA, 54--63.

Digital Library

[12]

Maria Alexandra Viegas Cortez da Cunha, Mônica Steffen Guise Rosina, Marco Antonio Carvalho Teixeira, Alexandre Pacheco da Silva, Eduardo Alves Lazzari, Maria Camila Florêncio da Silva, Rodrigo Moura Karolczak, Stefania Lapolla Cantoni, Taiane Ritta Coelho, Thomaz Anderson Barbosa Silva, Larissa Spinola, Lucas Marinho, and Nina Rentel Scheliga. 2015. Dados abertos nos municípios, estados e governo federal brasileiros. Technical Report. http://bibliotecadigital.fgv.br/dspace/handle/10438/16373

[13]

Li Ding, Timothy Lebo, John S. Erickson, Dominic DiFranzo, Gregory Todd Williams, Xian Li, James Michaelis, Alvaro Graves, Jin Guang Zheng, Zhenning Shangguan, Johanna Flores, Deborah L. McGuinness, and James A. Hendler. 2011. TWC LOGD: A portal for linked open government data ecosystems. Web Semantics: Science, Services and Agents on the World Wide Web 9, 3 (Sept. 2011), 325--333.

Digital Library

[14]

John S. Erickson, Eric Rozell, Yongmei Shi, Jin Zheng, Li Ding, and James A. Hendler. 2011. TWC International Open Government Dataset Catalog. In Proceedings of the 7th International Conference on Semantic Systems (I-Semantics '11). ACM, New York, NY, USA, 227--229.

Digital Library

[15]

J. S. Erickson, A. Viswanathan, J. Shinavier, Y. Shi, and J. A. Hendler. 2013. Open Government Data: A Data Analytics Approach. IEEE Intelligent Systems 28, 5 (Sept. 2013), 19--23.

Digital Library

[16]

Esri 2015. Independent Report Highlights Esri as Leader in Global GIS Market. (2 March 2015). Retrieved 2018-01-06 from http://www.esri.com/esri-news/releases/15-1qtr/independent-report-highlights-esri-as-leader-in-global-gis-market

[17]

Roy T. Fielding, Tim Berners-Lee, and Henrik Frystyk. 1996. Hypertext Transfer Protocol - HTTP/1.0. Technical Report. https://tools.ietf.org/html/rfc1945

[18]

ISO 2013. ISO 3166-1:2013 - Codes for the representation of names of countries and their subdivisions - Part 1: Country codes. (2013). Retrieved 2018-01-03 from https://www.iso.org/standard/63545.html

[19]

Ryan Mitchell. 2015. Web Scraping with Python. Collecting Data from the Modern Web. O'Reilly. http://shop.oreilly.com/product/0636920034391.do

Digital Library

[20]

Sebastian Neumaier, Jürgen Umbrich, and Axel Polleres. 2016. Automated Quality Assessment of Metadata Across Open Data Portals. J. Data and Information Quality 8, 1 (Oct. 2016), 2:1--2:29.

Digital Library

[21]

Sebastian Neumaier, Jürgen Umbrich, and Axel Polleres. 2016. Automated Quality Assessment of Metadata Across Open Data Portals. J. Data and Information Quality 8, 1, Article 2, 29 pages.

Digital Library

[22]

Open Data Watch. 2016. The Open Data Inventory 2016 Annual Report: Toward an open data revolution. Technical Report. Open Data Watch. http://odin.opendatawatch.com/Downloads/otherFiles/ODIN-2016-Annual-Report.pdf

[23]

Edobor Osagie, Waqar Mohammad, Arkadiusz Stasiewicz, Islam Ahmed Hassan, Lukasz Porwol, and Adegboyega Ojo. 2015. State-of-the-art Report and Evaluation of Existing Open Data Platforms. Technical Report 645860 H2020-INSO-2014. http://routetopa.eu/

[24]

Leonard Richardson and Sam Ruby. 2007. RESTful Web Services (1 ed.). O'Reilly Media, California, USA.

Digital Library

[25]

D. S. Sayogo, T. A. Pardo, and M. Cook. 2014. A Framework for Benchmarking Open Government Data Efforts. In 2014 47th Hawaii International Conference on System Sciences. 1896--1905.

Digital Library

[26]

Joshua Tauberer. 2014. Open Government Data: The Book - Second Edition. (2014). Retrieved 2014-11--18 from https://opengovdata.io/

[27]

Barbara Ubaldi. 2013. Open Government Data: towards empirical analysis of open Government Data Initiatives. OECD Working Papers on Public Governance. Organisation for Economic Co-operation and Development, Paris. Retrieved 2014-11--18 from

[28]

Atz Ulrich, Heath Tom, and Fawcett Jamie. 2015. Benchmarking open data automatically. Technical Report ADI-TR-2015-000. Open Data Institute. https://theodi.org/guides/benchmarking-data-automatically

[29]

United Nations Publications. 2016. United Nations E-Government Survey 2016: E-Government in Support of Sustainable Development. United Nations, New York. http://workspace.unpan.org/sites/Internet/Documents/UNPAN97453.pdf

[30]

Anneke Zuiderwijk and Marijn Janssen. 2014. The Negative Effects of Open Government Data - Investigating the Dark Side of Open Data. In Proceedings of the 15th Annual International Conference on Digital Government Research (dg.o '14). ACM, New York, NY, USA, 147--152.

Digital Library

Cited By

Ali MPapageorgiou GAziz ALoukis ECharalabidis YAlexopoulos CPellicer F(2024)A Framework for the Multi-Dimensional Assessment of Interoperability for Open Data Ecosystems DevelopmentInformation Polity10.1177/15701255241297172Online publication date: 16-Dec-2024
https://doi.org/10.1177/15701255241297172
Aziz AJosé Herrera-Murillo DNogueras-Iso JLacasta JLopez-Pellicer F(2024)Identifying the Evolution of Open Government Data Initiatives and Their User EngagementIEEE Access10.1109/ACCESS.2024.341428212(84556-84566)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3414282
Ahmed UAlexopoulos CPiangerelli MPolini A(2024)BRYT: Automated keyword extraction for open datasetsIntelligent Systems with Applications10.1016/j.iswa.2024.20042123(200421)Online publication date: Sep-2024
https://doi.org/10.1016/j.iswa.2024.200421
Show More Cited By

Index Terms

Investigating open data portals automatically: a methodology and some illustrations
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Information extraction

Recommendations

Laying the foundations for benchmarking open data automatically: a method for surveying data portals from the whole web
dg.o '19: Proceedings of the 20th Annual International Conference on Digital Government Research

The widespread of open data portals in the global scenario makes automated methods for data gathering and assessment a landmark in benchmarking. Some practitioners have already introduced automated approaches focused on mainly quality assessment of ...
Maturity and usability of open data in North Rhine-Westphalia
dg.o '18: Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age

We examine the actual state of open government in North Rhine-Westphalia (NRW), Germany, regarding the infrastructure for publishing open data. The overall state is measured with the Open Data Barometer. Furthermore, we analyse the distribution of file ...
Similarities of open data and open source: impacts on business

What are the similarities of open data and open source software when building a business? Despite their differences as phenomena (one is about applications and one is about data), the two also have many similarities. Both for example share the idea that ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

dg.o '18: Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age

May 2018

889 pages

ISBN:9781450365260

DOI:10.1145/3209281

Conference Chairs:
Marijn Janssen
Delft University of Technology, Netherlands
,
Soon Ae Chun
City University of New York (CUNY), USA
,
Vishanth Weerakkody
Bradford University, UK
,
Editors:
Anneke Zuiderwijk
Delft University of Technology, Netherlands
,
Charles C. Hinnant
Florida State University, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

CNPq

Conference

dg.o '18

dg.o '18: 19th Annual International Conference on Digital Government Research

May 30 - June 1, 2018

Delft, The Netherlands

Acceptance Rates

Overall Acceptance Rate 150 of 271 submissions, 55%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
265
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ali MPapageorgiou GAziz ALoukis ECharalabidis YAlexopoulos CPellicer F(2024)A Framework for the Multi-Dimensional Assessment of Interoperability for Open Data Ecosystems DevelopmentInformation Polity10.1177/15701255241297172Online publication date: 16-Dec-2024
https://doi.org/10.1177/15701255241297172
Aziz AJosé Herrera-Murillo DNogueras-Iso JLacasta JLopez-Pellicer F(2024)Identifying the Evolution of Open Government Data Initiatives and Their User EngagementIEEE Access10.1109/ACCESS.2024.341428212(84556-84566)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3414282
Ahmed UAlexopoulos CPiangerelli MPolini A(2024)BRYT: Automated keyword extraction for open datasetsIntelligent Systems with Applications10.1016/j.iswa.2024.20042123(200421)Online publication date: Sep-2024
https://doi.org/10.1016/j.iswa.2024.200421
Sampaio IAndrade EBernardini FViterbo J(2022)Assessing the Quality of Covid-19 Open Data PortalsElectronic Government10.1007/978-3-031-15086-9_14(212-227)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-15086-9_14
Kirstein FBacher DBohlen VSchimmler S(2021)Ronda: Real-Time Data Provision, Processing and Publication for Open DataElectronic Government10.1007/978-3-030-84789-0_12(165-177)Online publication date: 7-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-84789-0_12
Addone ADe Donato RPalmieri GPellegrino MPetta AScarano VSerra L(2020)Visual Storytelling by Novelette2020 24th International Conference Information Visualisation (IV)10.1109/IV51561.2020.00126(723-728)Online publication date: Sep-2020
https://doi.org/10.1109/IV51561.2020.00126
Correa AMelo Jr. ASilva F(2020)A deep search method to survey data portals in the whole web: toward a machine learning classification modelGovernment Information Quarterly10.1016/j.giq.2020.10151037:4(101510)Online publication date: Oct-2020
https://doi.org/10.1016/j.giq.2020.101510
Sheffer Correa ASoares Correa Da Silva F(2019)Laying the foundations for benchmarking open data automaticallyProceedings of the 20th Annual International Conference on Digital Government Research10.1145/3325112.3325257(287-296)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3325112.3325257
Fallucchi FPetito MDe Luca E(2019)Analysing and Visualising Open Data Within the Data and Analytics FrameworkMetadata and Semantic Research10.1007/978-3-030-14401-2_13(135-146)Online publication date: 24-Feb-2019
https://doi.org/10.1007/978-3-030-14401-2_13

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents