1 Introduction
Nowadays, many state and local governments are paying close attention to providing citizens and employees with accessible solutions to the physical environment. However, many organizations still have not completely complied with accessibility obligations through their digital properties even if the Internet is dramatically changing the way that they have to serve the public. Taking advantage of new technology, it is possible to use the web for offering citizens a host of services including: (1) corresponding online with local officials; (2) providing information about government services; (3) renewing identity cards; (4) providing tax information and accepting tax returns; and (5) applying for jobs or benefits. In the near future, people may not have a choice when accessing a public sector and they will necessarily have to use a website or mobile app, so it is important they work for everyone. Even more, if we take into account citizens with disabilities, local government websites have to allow citizens with visual, auditory, and other physical limitations and disabilities to access their digital content. Therefore, websites must be optimized to work in conjunction with assistive technology otherwise, the websites will be a sort of barrier for disabled citizens, limiting, or completely inhibiting, their ability to obtain all available information from the website.
The goal of this paper is to investigate accessibility issues that affect municipality websites. Accessible features can be used not only to ensure access to people with disabilities but also to make webpages more usable both by people using older computers and by people using the latest technologies (such as personal digital assistants, handheld computers, or web-enabled cellular phones). This rule is also regulated at the legislative level. Regulations such as the
Americans with Disability Act (ADA) [
1] or the EU Web Accessibility Directive [
2], say how public websites or applications have to be accessible for visitors of all abilities.
The main research issue we want to study in this paper concerns the accessibility challenges a local public organization has to deal with when it has to provide services through a so-called e-government portal. To guarantee e-government accessibility means (1) identifying the better strategies to evaluate how far a website is from specific accessibility requirements; (2) analyzing the methodologies used to conduct and validate such researches; and (3) determining the fundamental effects of these studies on the design of websites.
In detail, we are interested in studying the impact of the local government websites in terms of applicable standards and regulations. The municipality websites are the first access point the citizen has to use in accessing e-government services. At the current stage, have municipalities made an effort to adapt official websites to the citizen's needs? What actions can be taken to improve the accessibility of local government websites?
To study these problems we took into account specifically Italy's case. Our study highlights how Italy is a typical example of a country in which e-government has received little attention. The few studies that have been conducted in Italy call for greater concern for improving the quality of services to citizens and speeding up internal government procedures.
Generalizing the results of the detailed analysis we carried out in Italy's case, we will enable researchers to identify the research gaps in terms of foci in e-government accessibility and the most appropriate methodology and theoretical lens to adopt in a bid to holistically resolve e-government accessibility issues.
Then a question arises: What do we have to do for assuring web accessibility of municipality websites?
The paper presents an analysis procedure so defined: (1) Identification of the legislative framework to which municipality websites must adhere; (2) Research of a better tool that can be used to carry out automatic detection of the accessibility problems that affect the municipality websites; (3) Analysis of the accessibility problems and visualization and correction, if possible, of the detected problems.
For identifying the legislative framework to take into account, we need to consider the international theoretical framework is defined by the
Web Content Accessibility Guidelines (WCAG) international standards [
3], according to which local accessibility guidelines [
4] are stemmed and defined. For example, taking again the Italian case study, the law known as the Stanca Act [
5], relying on the WCAG standard aims to support access to information technologies for the disabled and addresses accessibility considerations in Italy.
Once we have defined the theoretical and legislative framework, we need a practical methodology to identify what accessibility issues are present on a website before fixing them. To do it, it is possible to use an accessibility validator that is a tool that can provide automated website testing that strictly adheres to WCAG standards or other local accessibility guidelines.
On the Internet, several validators can be discovered but we need to identify the better one that can be used to analyze municipality websites of a given state. For example, in our case study, starting from the assessments we presented in [
6], Section
2 reports an investigation of the most used accessibility tools that allows us to find the most useful tools for evaluating sites in the context of Italian law.
In detail, we analyzed two validators, AChecker [
7] and VaMolà [
8], to establish the best tool for identifying problems on a web page according to the Italian law. In detail, by using such tools, we analyzed the websites of all Italian municipalities for a total of 7,713 homepages of public administration according to two accessibility guidelines: Stanca Law – Annex A L.4/04 and WCAG 2.0 - Level AA. Taking into account both validators, we carried out 30,852 evaluations. From the obtained results, it emerged that the VaMolà validator, unlike AChecker, prepares periodic updates about the evaluation parameters of the accessibility guidelines and so it is more useful for the validation of the municipality pages. Furthermore, by comparing specific types of errors in the two validators, VaMolà has shown a greater ability to distinguish different categories of problems, allowing a more precise evaluation of the accessibility rate of the pages. The VaMolà validator has been therefore selected as a validation tool to carry out the last step described in this paper.
Finally, in the last phase of our analysis, we need a strategy to visualize and possibly correct the detected accessibility problems. As we will demonstrate for the Italian case study, for most websites, implementing accessibility features is not difficult and will seldom change the layout or appearance of web pages. In this phase, we designed and developed a website that allows any municipal employee, even non-technical, to search a municipality website, to examine the errors by using VaMolà, and finally to visualize and correct, if possible, the detected accessibility problems.
The remainder of this paper is organized as follows. Section
2 presents the main work related to the evaluation of websites’ accessibility and a description of the methodology applied in our study. In Section
3, we describe the main issues of our accessibility study based on the use of two validators: AChecker and VaMolà. Section
4 presents the web application we developed for helping municipal officers to detect and correct accessibility problems in their websites. Section
5 concludes the paper.
2 Background and Related Works
In this period of a pandemic due to the COVID-19 outbreak, public agencies such as municipalities can offer a very limited set of services specifically when they must be served in the actual presence of the person. For this reason, the accessibility of digital content has important implications for citizens whenever public institutions have to be compliant with the law in force.
Electronic government (or e-government) is the application of Information and Communication Technologies (ICTs) to government functions and procedures with the purpose of increasing efficiency, transparency, and citizen participation.
A first sub-section of this literature review of related works focuses on investigating how accessibility becomes more crucial as governments advance in the provision of online services. Failure to resolve accessibility issues will create a sort of “digital disablement”.
A second subsection describes how this problem is dealt with in Italy. The Italian disabilities act requires that state and local governments provide citizens with equal access to their websites. The Stanca Act states that the government protects each person's right to access all sources of information and services independent of disability. The law clarifies and extends the rights of the disabled to access public services via the Internet in line with the principles of equality established in Article 3 of the Italian Constitution.
A third subsection aims at studying procedures of accessibility validation to do preliminary assessments of the e-government websites.
Specifically, the paper focuses on studying automatic evaluation tools that help with evaluation. However, no tool alone can determine if a site meets accessibility standards. Knowledgeable human evaluation is required to determine if a site is accessible. Nevertheless, evaluating accessibility early to identify accessibility problems early is better than dealing with them later.
Finally, the last subsection is dedicated to presenting accessibility-testing tools specifically addressed to analyze the Italian municipality websites.
2.1 Related Works on E-Government
Studies carried out over recent years tend to evaluate the user satisfaction with e-government portals [
9] but more is necessary to determine if and how they enhance public information accessibility. Authors in [
9] claim how there is increasing trustworthiness among citizens in e-services as it is safer if compared with face-to-face services, but this confidence needs to be repaired with a service that complies with accessibility standards.
Some other studies such as the one presented in [
19] focused on e-government satisfaction evaluation whereas other studies such as the ones described in [
10–
17] are more oriented to check the existence of accessibility issues for e-government sites, even those whose governments claimed adherence to accessibility standards or legislation.
Studies published in [
10–
19] examine the accessibility of e-government websites of different countries. In detail, [
18] reports an analysis of accessibility standards and guidelines based on a regulatory framework. One of the first and most well established examples of such government regulations is Section 508 in the USA [
14]. Section 508 amended in 1998 the Rehabilitation Act of 1973. It requires all federal agencies to design all information technology accessible to disabled users. Other countries reference national standards such as France with the RGAA 3, Germany with the BITV, the Netherlands with Dutch guidelines, Ontario with the SGQRI 008, and Spain with the standard UNE 139803:2012. By analyzing these regulations it can be seen that they stem or are an extension of the
WCAG (Web Content Accessibility Guidelines) [
20], which is an international standard recognized as the benchmark for web accessibility. For example, concerning WCAG 2.0, Section 508 is expected to reach a Level AA conformance [
14].
The WCAG is an international standard since 2012, via the ISO/IEC 40500:2012 (Information technology –W3C Web Content Accessibility Guidelines) and it is referenced in the regulations of several countries that mandate accessibility of public websites such as USA, Australia, Canada, Italy, Norway, and the United Kingdom.
Other research studies [
17,
21,
22] have been conducted on e-government in developing countries highlighting how e-government is necessary for improving governance, especially in developing countries. Nevertheless, these studies remark how the accessibility challenge is a hindrance that makes e-government services difficult to reach all citizens. For example [
19] states how accessibility issues have resulted in an even higher digital divide and disparity in e-government service provision between urban and rural cities. Other e-government accessibility studies have focused mostly on developing countries reporting how they receive even less attention and care, particularly in Africa [
22].
Because of these research studies, most e-government projects remain unsuccessful in both developed and developing countries. If e-government accessibility remains a challenge, it is in the best interest of researchers to interrogate how they have to face and investigate the problem.
As said before, the fundamental starting point of this investigation is the work carried out by the
World Wide Web Consortium (W3C), the main international standards organization for the World Wide Web. In this regard, W3C has developed the WCAG to make the web accessible to people with disabilities. Specifically, in our study, we take into account the WCAG 2.1, the latest version of these guidelines that covers a wide range of recommendations for making web content more accessible [
23].
Following these guidelines, governments can make the content of their websites more accessible to a wider range of people with disabilities, including accommodations for blindness and low vision, deafness, limited movement, speech disabilities, and combinations of these, and some accommodation for learning disabilities and cognitive limitations [
24,
25,
26]. People with disabilities can use websites when they are designed and coded appropriately. However, public sector bodies continue to develop websites with accessibility barriers that make it difficult to be used both for people with disabilities and people without these apparent difficulties, although, according to the W3C, making the web accessible “benefits individuals, businesses and society”.
This approach, known as the “Design-for-all strategy”, is of particular value for information society technologies, which change at a very high speed. It is now important to convince the potential players to take up the concept for implementation according to their responsibilities: politicians, administrators, company leaders, developers, engineers, designers, educators, and association leaders constitute only examples of the group.
2.2 Italian Case Study
As a case study in this paper, we take into account what happens in Italy. The impact that e-government can have is that of a better public administration, in that it allows public policies to be optimized, the quality of services to be raised, the involvement of the citizens to be widened, and other specific fundamental activities to be improved. Nevertheless, Italy is a typical example of a country that although it has a well-defined accessibility law in the Stanca Act, the vast majority of its municipality websites do not comply with it.
In Italy,
AgID (Agenzia per l'Italia Digitale - Agency for digital Italy) [
27] promotes website accessibility concerning the Law 4/2004 - legge Stanca (Stanca Act). The Stanca Act is an Italian law of 2004 that promotes the accessibility of information technology. The law applies also to the Italian government websites.
AgID is the technical agency of the Presidency of the Council of Ministers and according to the provisions of current legislation, it is responsible for monitoring the websites of the public administrations and providing assistance to meet the law requirements. Specifically, AgID aims at supporting public sector bodies in the design of websites able to provide citizens with IT services without discrimination, also by individuals with disabilities who need assistive technologies or special configuration (art. 2(a) Law 4/2004). The websites of public administrations must comply with the technical accessibility requirements specified in Annex A of the Ministerial Decree of 8 July 2005, as amended.
While public administration’s top priority is undoubtedly the equitable access of content by all citizens, it is important to understand that local governments that fail to comply with accessibility standards could face a legal and financial penalty in many countries especially in North America and Europe. For example, the Web Accessibility Directive (Directive (EU) 2016/2102) [
2] has been in force since 22 December 2016 and provides people with disabilities with better access to websites and mobile apps of public services. The Directive obliges websites and apps of public sector bodies to meet specific technical accessibility standards.
Although Italy has legal protection to ensure equal access to e-government websites, the mere presence of a law does not guarantee compliance. This paper aims at studying the current situation in Italy and at evaluating how much the Italian municipality websites are compliant with the law in force.
2.3 Accessibility Validation Methodologies
The increasingly recognized importance of accessibility implies that various stakeholders, with different expertise, look at it from different viewpoints and have different requirements regarding the types of outputs they expect.
For this reason, it is very important to keep in mind that no website evaluation tool can completely replace a human being. This is because with present technology it is difficult to emulate human attributes such as common sense. Moreover, since accessibility is a subset of usability, these tools should only be used to evaluate accessibility and not usability since, at best, they can only show you where your site is not accessible.
Nevertheless, although to determine if a site is accessible according to a given guideline, a knowledgeable human evaluation is required, the technologies used to support Web accessibility evaluation are evolving along with the associated accessibility guidelines. Examples of these technologies are the automatic evaluators, which are tools that test web applications by analyzing the code to verify if those are in conformity with the accessibility guidelines selected for the inspection. For example, these validators could be used to test the presence of alternative texts of the images or could verify the algorithms for calculating appropriate web color combinations before going on with further validation to match the requirements of the users with different vision deficiencies.
W3C provides us with a constantly updated Web Accessibility Evaluation Tools List at this link:
www.w3.org/WAI/ER/tools. This list includes information on more than 100 tools that you can filter to narrow down the list to the types of tools you are interested in. These Web accessibility evaluation tools can help you quickly identify potential accessibility issues. For example to achieve WCAG compliance several tools turned out to be very helpful: Firefox Web Developer Toolbar, Lynx, Web Accessibility Toolbar, W3C-Validators (XHTML + CSS), Cynthia, WebXact, Taw, Mauve, AChecker, any browser and screen readers such as Jaws [
28,
29]. Although, automated testing tools are not always appropriate and have certain limitations (e.g. changes in the natural language, consistent and usable structure and navigation, etc.) they can provide useful indications about the accessibility level of different kinds of websites (i.e. governmental, public administration, higher education [
28]) according to national or international guidelines. These tools can carry out automatic tests for providing a meaningful and structured screening of accessibility level and trend of municipality websites. These tools could also represent a valuable solution to support developers and those who are required to make decisions, especially at the governmental level. In the next section, we present a set of these validators for improving the quality and accuracy of automatic and semi-automatic support to web accessibility assessments, detecting accessibility hurdles and assisting them in repairing accessibility barriers. In particular, we analyze these solutions for what concerns their use in achieving the Italian accessibility law compliance.
Other studies tried to carry out a similar analysis. Two examples are reported in [
30,
31,
32] where institutional webpages of some Italian municipalities and universities have been analyzed using accessibility evaluation tools. In particular, in [
31,
32] the authors present an overview of the Italian situation and report a study based on the analysis of websites of the chief towns of Italian provinces (approximately one hundred). Even if this work does not perform a complete analysis of the Italian municipalities, it is sufficient for claiming that the Italian institutional websites considered are not accessible. This is because they do not even pass the syntax test (carried out with the accessibility evaluation tool), which should be a prerequisite for accessibility. In the following, we propose a more detailed study carried out on all institutional websites of all 7,713 Italian municipalities. This result has shown a variety of accessibility problems with the sites, but we can demonstrate how most of these issues are centered on a minority of specific checkpoint errors, such as the lack of providing alternate text for images. Therefore, we can conclude our study suggesting for web developers a solution for implementing design recommendations able to improve the accessibility rankings of their sites and provide sites that are more open to people with disabilities.
2.4 Accessibility Testing Tools for Analyzing Italian Municipality Website
The web Accessibility evaluation tools list published by the W3C at this link
https://www.w3.org/WAI/ER/tools/ and in [
28,
29], presents a collection of accessibility checkers used to evaluate the web accessibility in different domains.
Despite the popularity of automated web accessibility evaluation tools in practice, few studies focus on comparing the performance and quality of web accessibility evaluation tools specifically in evaluating Italian e-government websites.
There are many considerations to take into account when selecting an evaluation tool. Web designers could be looking for tools to analyze their design's accessibility performance. In our case, we need to select tools that help to assess the code of the Italian municipality websites against the Stanca Act.
Starting from an analysis of the main tools provided by the W3C, we identified a set of six validators as possible candidates to check the homepages of Italian municipality websites: Taw [
33], Cynthia [
34], AChecker [
7], VaMolà [
8], Wave [
35], and Mauve [
36]. We selected these tools according to their popularity and their capability to perform validation in conformance with the Italian accessibility guidelines.
After a careful test of all validators on a sample of six websites, it emerged that the Taw and Mauve validators correctly validated the selected web pages without providing further suggestions about how to fix the detected errors. These two validators cannot be used to design a web application that can help municipal officials to fix their websites. This test highlighted how Cynthia and Wave allow the validation of the pages through the WCAG 2.0 and Section 508 guidelines but they cannot be used to perform a validation according to the Stanca Law guidelines. AChecker and VaMolà are validators able to validate web pages against the Stanca Law guidelines. Moreover, they are also able to indicate the proper “row” and “column” in which the violation is present.
AChecker (Accessibility Checker) is an open-source accessibility evaluation tool developed by the Inclusive Design Research Centre of the University of Toronto. Using this tool, it is possible to submit a web page via its URL or by uploading its HTML file and subsequently select which guidelines to evaluate it against, namely the HTML Validator, BITV, Section 508, Stanca Act, WCAG 1.0 and WCAG 2.0. AChecker identifies three types of errors: Certain; Probable; and Potential problems. The latter is a type of error that AChecker cannot identify and that require manual intervention.
VaMolà (standing for “Accessibility Validator and Monitor”) Validator is open-source software, based on AChecker, available as a Web service and as a Web-based application. Such a validator automatically checks a single URL and it allows you to specify which accessibility guidelines the evaluation has to control, in particular for our scope, the Stanca Act. In January 2014 the validator was updated to the latest version of the technical requirements of the Decree of 20 March 2013, published in the Official Gazette in September 2013. The validator has a very simple interface, is completely in Italian, and gives the user the possibility to consult a synthetic report in which to quickly view the errors. VaMolà classifies errors into two categories: Certain and probable problems. The first are errors that for sure are barriers to accessibility. The second type of error are problems that have been identified as probable barriers, but require the manual control of a human to make a final decision.
3 Research Methodology
3.1 Domain Study: Accessibility Analysis of Homepages of Public Administrations
Satisfaction experiences when users interact with e-government websites hinge on the extent to which the information and services being offered are user-centered and in particular citizen-centered [
37]. To test the usability of such websites, as reported in [
38] researchers are continuing to develop new approaches to more and more effective studies. According to these studies, several reasons could deter users to use e-government websites. Some examples could be the lack of interactivity, lack of availability of services or information that people really want, or the overload of information that these websites provide. Although some consider accessibility testing a subset of usability, in this paper, we focus our study on the urgency of making universal accessibility an increasing issue in e-government.
This is because the main issue of many e-government websites is that they are not always designed to be in the first instance accessible. Web accessibility means that websites, tools, and technologies are designed and developed so that people with or without disabilities can perceive, understand, navigate, and interact with digital content.
Website Accessibility Evaluation Methodology (WCAG-EM) [
39] provides a complete methodology for the evaluation of websites for WCAG 2.0 conformance. WCAG-EM provides a methodology that covers different situations. Regardless of the context of use, the conformance evaluation procedure is detailed under five main steps: (i) Define the scope of the evaluation; (ii) Explore the website; (iii) Select a representative sample; (iv) Evaluate the selected sample; and (v) Report the evaluation findings.
By following this methodology, the paper focuses on evaluating the web accessibility of the sites of Italian municipalities. We examined 7,914 websites according to the updated number of Italian municipalities declared by ISTAT as of 31 December 2019 [
40]. Only the “homepages” of all municipal sites have been analyzed since analyzing the entire site for each Italian municipality would have been too expensive in terms of time and resources due to a large amount of data. Moreover, if the doorway that is the homepage is not accessible, the rest of the site will not be consulted as well.
Starting from the accessibility assessment carried out in the previous accessibility study of 2016/2017 [
6], the purpose of this work is to compare the results of the previous analysis with the current one to examine improvements or worsening in the accessibility of the sites, also making use of the use of a second validator: VaMolà. The final goal is to provide a web application that allows a user to view the homepage of the requested municipality, to recognize and possibly resolve the accessibility errors present in it.
To extrapolate comparable data from the analyses carried out by both AChecker and VaMolà, we developed a script, based on the Java Jsoup library [
41], for each validator to identify the actual number of errors of each municipality homepage, dividing them between known and likely errors.
Known problems are accessibility barriers that need to be removed to make the website accessible according to the adopted regulation. Likely problems are probable barriers that need a human evaluation for making a decision. Moreover, AChecker can detect potential problems that are problems that cannot be identified without human intervention.
These potential errors of AChecker do not find direct correspondence with the types of errors that VaMolà can detect.
To understand the characteristics of the potential problems detected by AChecker and eventually to find a counterpart in those detected by VaMolà, we carried out an analysis on a sample set of a hundred municipality homepages. By this analysis, it emerged that all potential errors detected by AChecker have been classified as likely errors by VaMolà. For this reason, to compare the results of the two validators on all municipality homepages, AChecker probable and likely errors have been merged.
From a technical point of view, the script we developed reads a file containing the list of the URL of each municipality provide by the ISTAT. After establishing the connection with AChecker and VaMolà validator's websites, the script takes each URL and parsed the related homepage. The last step is to save the web page resulting from the validation in a local database. While downloading the pages analyzed by both validators, it emerged that some sites no longer exist. The reason can be found in the merging and suppressing of some municipalities carried out by the government during 2019. For this reason, the final number of municipalities on which accessibility assessments were carried out were 7,713 instead of 7,914.
The comparison of results carried out by the two validators highlighted how the detected problems were classified according to different requirements. AChecker uses the 22 requirements of the first draft of 2004 of the Stanca Act, whereas VaMolà uses the 12 requirements of the last version of the Law of 2013. Technically, the 2013 ratification reduced the analysis requirements from 22 to 12 checkpoints. This numerical reduction does not correspond to a reduction of the rules to be controlled but to a redefinition of a more reasoned list of requirements that can be more compliant with the directives of WCAG 2.0 - Level AA conformance, from which the Stanca Act stems. This prevents a direct comparison between the results of the two validators about the Stanca Act.
3.1.1 Result of Comparing Web-Accessibility Validators Against WCAG 2.0 Requirements.
The first part of this study was conducted by testing the accessibility conditions of all Italian municipalities’ websites against the WCAG 2.0 guidelines by using both the VaMolà and AChecker validator. According to the VaMolà validator, 12% of the analyzed homepages were free of certain errors while only 5% were free of potential errors. AChecker showed similar results although the number of pages containing “certain errors” decreases slightly instead the number of “probable errors” increases as shown in Figure
1.
These results suggest that AChecker detects more errors than VaMolà in particular for the likely errors. Conversely, for those considered as the known problems VaMolà detects a slightly higher number of errors than AChecker.
To identify possible anomalies in the recognition of errors, we calculated the standard deviation of known and likely errors that occurred on the homepages of the municipalities of each region (Figure
2).
By analyzing the graphs in Figure
2, we can see a greater variation with respect to the average value for the likely errors detected by AChecker against a similar trend for known errors. To understand why AChecker and VaMolà detect a different number of errors and why for known errors AChecker presents a peak in correspondence of the Sardegna and Marche region, we carried out a further and more detailed analysis.
To this aim, we used a set of 12 requirements extracted by the WCAG 2.0 guidelines that are also at the base of the ratification of the Stanca Law assessment of 2013 (see Table
1). These technical accessibility requirements and the relative checkpoints have been used to verify the compliance of all municipality homepages.
The four guiding principles common to WCAG 2.0 and this list of requirements are the following:
•
Principle 1: perceptible - the information and components of the user interface must be presented in such a way that they can be used through different sensory channels.
•
Principle 2: usable - the components of the user interface and the commands contained therein must be usable without undue inconvenience or constraints for the user.
•
Principle 3: understandable - users must be able to understand how the interface works and the actions contained therein necessary to obtain services and information.
•
Principle 4: robust - the content must be robust enough to be interpreted reliably by a wide range of programs used by the user, including assistive technologies.
Compliance checkpoints refer to the WCAG 2.0 Success Criteria. Compliance with the following requirements corresponds to the AA compliance level of WCAG 2.0.
To do it, we developed a script to map each detected errors by VaMolà and AChecker on to the 12 requirements. We executed both validators to check the compliance of the homepages on the WCAG 2.0 (Level AA) guidelines. Then we associated each guideline at the level of compliance AA to one of our 12 requirements. Finally, we compared the results about problems detected by VaMolà and AChecker dividing them between known and likely errors. AChecker probable and likely errors have been merged.
Figure
4 shows that the most frequent known error detected by VaMolà is “Distinguishable” (ID req4) which represents 53% of the detected errors, followed by the “Textual alternatives” (ID req1) with 22% and the “Navigable” (ID req8) with 18%. AChecker presents a similar distribution but with a higher frequency for the errors of the type “ID req1” and a lower number of errors for the requirement “ID req8”.
We carried out a sample analysis on 40 homepages (two for each region) to check the most detected types of errors that contrast the requirements ID1, ID4, and ID 8. The ID req1: “Textual alternatives” specifies that not all non-textual contents shown to the user present an equivalent textual alternative that communicates the same intrinsic message as the non-textual content. The ID req4: “Distinguishable” reveals that color is used as the only visual mode to communicate information or as a visual distinguishing element. Alternatively, the contrast between foreground and background text is not at least 4.5: 1, the minimum contrast indicator specified by the W3C. Finally, the ID req8: “Navigable” indicates that some homepages or information blocks lack a title describing their purposes, or the focuses of the web elements needed to carry out sequential navigation, do not respect a correct order that preserves the sense and operation to facilitate the work of assistive technologies. The same types of errors have been detected by analyzing the likely errors but with differences in frequency. In this case, the most recurrent type of problems detected by VaMolà is related to the requirement “ID req8: Navigable", the percentage of which increases. On the contrary, the number of errors about the requirement ID req4 decreases whereas for the other types of errors the distribution is similar. Finally, for the likely errors, AChecker presents a very different distribution due to a high number of errors in the category ID req1: “Textual alternatives”. Moreover, we can note that the number of probable errors relating to the “Distinguishing” category is zero.
By analyzing these data, we can conclude that the AChecker validator does not make a distinction in terms of known or likely errors as regards the type of error ID req4. The error: “Distinguishable”, has always been classified as known and never as likely or potential. On the other hand, VaMolà can distinguish between known and likely errors in this category in a better way.
In conclusion, these analyses can allow us to understand that AChecker, besides using the old 22 requirements of the first draft of 2004 of the Stanca Act as revealed in the previous study, does not provide us reliable results also, for what concerns the WCAG 2.0 guidelines. It uses an insufficient number of parameters to distinguish a different severity of errors for all types of requirements provided by the WCAG 2.0 guidelines.
For this reason, since VaMolà uses the updated version of the Stanca Act, provides us with a better categorization of the different types of errors detectable by the WCAG 2.0 guidelines and the greater linearity in terms of reported results, we can claim that VaMolà is the best tool to detect accessibility problems.
Therefore, we chose to analyze the homepages of the municipality websites to discover accessibility problems with certainty according to the Stanca Law. Figure
4 presents the results of VaMolà highlighting how the absence of known problems and likely problems is very limited. This means that 10 years after its introduction, the government regulation is respected by only about 12% of the Italian municipalities and this is reduced to less than 5 % if we consider the probable errors. To better understand the VaMolà validation according to the Stanca Act guidelines, we can better analyze its results against the 12 requirements in Table
1 as reported in Figure
3. We better analyzed the types of errors that contrast the requirements ID1, ID4, and ID 8 by expanding the study on more homepages (four municipalities for each region with a total of 80 sites).
For both known and likely errors, VaMolà identifies a great number of problems concerning the capability to provide textual alternatives for any non-textual content (ID req1), the difficulty to separate the foreground content from the background (ID req4) and also problems related to the navigation of the homepages such as the absence of titles or misuse of focuses (ID req8).
Problems that can be fixed with a quite quick and easy intervention by municipality officers to obtain universal access to public local content in our country is described in the next section.
3.2 Domain Study: Design of a Website to Assist Public Administration Officers
The last objective of the present work aims at designing and developing a web application based on VaMolà that enables municipal officers who can have access to the code of the desired municipality, to identify and subsequently correct the accessibility issues detected on the page. As described in the previous sections, the VaMolà validator has been found as the most suitable tool for examining accessibility according to the Stanca Act. Moreover, VaMolà can identify the location of the detected error within the HTML code providing the reference of the line and column in which it is present. The validator also provides a textual description in Italian to better identify the error and offers a possible correction for detected accessibility errors.
The developed web application (that we named Verum) has a search bar to select the desired municipality and view its HTML code. By mapping the errors resulting from VaMolà on the source code, the web app allows officers to view the errors in the exact point of the code with an attached description of the error and a textual suggestion to how to correct it (in a pop-up window. See Figure
1). We carried out a test on the website of the Milano municipality. We parsed the homepage by using VaMolà and then we displayed the known and likely errors on the HTML code of the page in the Verum application. In Figure
5, three errors are detected in the HTML code and a pop-up window displays the type of problem. All three errors highlight the absence of the attribute “title” in the related tags
After the preliminary test on the webpage of the Milano municipality, we carried out an automatic detection on about one hundred websites. During the second test, some problems arose that prevented its complete realization. While identifying errors within the web page and providing a possible solution to the problem, VaMolà cannot correctly identify the location of the error within the HTML code of the page. By carrying out our sample check on about one hundred homepages, in more than half of these, we noticed how VaMolà identified all errors on the same line of code (see Figure
6).
Failure to recognize the location of the error forces us to carry out a manual search of the error line and column, using the text fragment provided together with the error itself. Moreover, VaMolà does not provide the whole fragment of text for each error detected, but only a preview (see Figure
6). The amount of work required to manually map a large number of pages would have been too much. In this regard, an alert was sent to the developers of VaMolà to report the bug.
4 Conclusions
The main research question this paper wants to answer concerns the accessibility challenges a local public organization has to deal with when it has to provide services through an e-government portal.
In detail, the paper describes a procedure aimed at identifying the legislative framework to which municipality websites must adhere; researching the better tool that can be used to carry out automatic detection of the accessibility problems; finally analyzing the discovered problems, and visualizing and correcting them, if possible. The carried out analysis has been focused on studying the Italian case. Municipalities in Italy made an effort for adapting their websites to the requirements indicated in the accessibility Italian Law. This law is commonly known as the “Stanca Act” after Lucio Stanca, who served as the Minister for Innovation and Technologies at the time of the act's passage.
Nevertheless, these efforts are not enough. Problems with content accessibility arise in the vast majority of websites as highlighted by the analysis described in the paper.
By following the methodology described in Section
3, we started our study by analyzing the theoretical and legislative framework to which each municipality website must adhere with particular attention to the Italian case.
Then we detailed a practical methodology to identify and analyze those problems that prevent these websites from offering democratic access to all citizens. For these analyses, we started by investigating the accessibility problems present in the homepages of the municipality websites to determine which is the most suitable validator in the detection of errors according to the requirements of the Stanca Act. We identified two validators suitable for the purposes of our study: AChecker and VaMolà. These validators brought out that most of the Italian municipalities are still in accessibility conditions that are not adequate to the standards defined by the Stanca Act and WCAG 2.0.
After a further study, we identified VaMolà as the most suitable validator to examine the municipality web pages for the following reasons
(1)
It uses the guidelines of the Stanca Act and presents a classification of the errors according to 12 categories specified in the last draft of the 2013 ratification;
(2)
VaMolà can detect a series of results without excessive error peaks on some municipalities, as happens instead with AChecker;
(3)
VaMolà has a better ability to distinguish in terms of the severity of errors for some requirements than AChecker. It does not present significant differences between the assessments of known errors and likely errors in the analyzes carried out by using the list of 12 requirements in Table
1;
(4)
VaMolà can identify the location of the error detected within the HTML code of the validated page, providing the reference of the line and column in which it is present to facilitate identification for any subsequent correction;
(5)
It also presents valid textual descriptions in Italian to better identify the error and already provides a possible correction for detected accessibility problems.
By analyzing the categories of errors with both validators, it has been noted that the majority of errors can be easily fixed. These changes do not require extensive code reworking and are easily correctable even by non-technical users.
To do it we developed a web application that allows non-technical personnel to change the code of the homepage of a given municipality. All this can be made possible through the implementation of textual suggestions and the correct positional identification of the error within the HTML code of the page in question.
Unfortunately, at the current stage, VaMolà presents a bug that prevents us from exactly identifying the position of the error in the HTML code leaving us unable to carry out our objective. Future works aim at collaborating with the VaMolà developer for fixing this bug and for finally developing and testing on the ground web applications.
Once this bug is fixed and despite the limitations due to the pandemic situation, we are ready to plan real user tests through the involvement of some local municipalities in the Lombardia region.
To conclude, we can claim that our study, carried out for the Italian case, can be generalized for all local government websites spread over the word. First, the methodology we adopted to identify the best validator to use for checking accessibility issues of the websites, can be replicated for each country.
In the selection of the validator, we need to take into account whether it is able to control the sites according to the law in force in the country under consideration. Then, it is necessary to verify that the validator is able to identify where the accessibility problem is located in order to guide to its correction at the HTML code level. Once the best validator is identified, we can use our script to detect the actual number of errors on each municipality homepage, dividing them between known and likely errors. The last step of our methodology aims at using an extension of our web application to enable municipal officers who can have access to the code of the desired municipality, to identify and subsequently correct the accessibility issues detected on the page.
Our hope is that the application of our methodology will provide local governments with a strategy to improve website availability. A repetition of the study in other countries in a few years will enable the observation of trends in the improvement of the local government websites’ accessibility.
Acknowledgments
This work stems from a previous thesis: “Accessibility in the public administration” [
6]. The authors wish to thank all students involved in this study for their useful works.
