[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Next Article in Journal
Creative Talent and Personality: A Primary Education Study
Next Article in Special Issue
Online Learning Tools in the Era of m-Learning: Utility and Attitudes in Accounting College Students
Previous Article in Journal
Population and Economic Projections in the Yangtze River Basin Based on Shared Socioeconomic Pathways
Previous Article in Special Issue
Mobile Apps for Helping Patient-Users: Is It Still Far-Fetched?
You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 12
Issue 10
10.3390/su12104201
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Developing and Exploring an Evaluation Tool for Educational Apps (E.T.E.A.) Targeting Kindergarten Children

by
Stamatios Papadakis
1,*,
Julie Vaiopoulou
2,3,
Michail Kalogiannakis
1 and
Dimitrios Stamovlasis
4
1
Department of Preschool Education, Faculty of Education, University of Crete, 74100 Rethymno, Greece
2
Department of Primary Education, Faculty of Education, Democritus University of Thrace, 57001 Thessaloniki, Greece
3
School of Education, University of Nicosia, CY-1700 Nicosia, Cyprus
4
Department of Philosophy and Education, Faculty of Philosophy, Aristotle University of Thessaloniki, 57001 Thessaloniki, Greece
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(10), 4201; https://doi.org/10.3390/su12104201
Submission received: 8 April 2020 / Revised: 28 April 2020 / Accepted: 16 May 2020 / Published: 20 May 2020
(This article belongs to the Collection Sustainability in the Era of Mobile Learning)
Versions Notes

Abstract

:
During the last decade, there has been an explosive increase in the number of mobile apps that are called educational and target children aged three to six. Research has shown that only a few of them have been created taking into consideration young children’s development and learning processes. The key question that emerges is how parents, custodians, or teachers can choose appropriate, high-quality educational apps. Literature has presented limited assessment tools based on advanced statistical procedures, which allow one to address validity and reliability issues. This study investigates the dimensions of using and operating educational apps for kids and presents a thirteen-item assessment instrument along with its psychometric properties. Data (N = 218) were collected via an electronic questionnaire from pre-service teachers of preschool education. Principal Component Analysis (PCA) with Varimax rotation was used to investigate the underlying dimensions. The resulting structure included four factors, namely: Usability, Efficiency, Parental Control, and Security. PCA supported the factorial validity of the instrument, while the reliability measures of Cronbach’s alpha for the four dimensions were satisfactory. Finally, a lucid discussion on the findings is provided.

1. Introduction

Although interactive touchscreen technology made its public appearance in 2007 with the introduction of the iPhone from Apple, one of the first mainstream multimedia devices, it was the release of the first tablet-type device (Apple iPad) on April 3, 2010 that so profoundly changed the ecosystem of smart mobile devices. In terms of sales, the first iPad sold more than 300,000 units in its first 24 hours, and almost 15 million units in nine months. For the first time, the general public had access to a high-quality device with a crisper screen, eliminating the ergonomic problems that users encountered with the smaller screens of mobile phones and other devices such as Personal Digital Assistants (PDAs) [1].
The popularity of these devices is due, in part, to the fact that they were soon seen by the general public as an alternative to the ‘traditional’ graphical user interface (GUI) of desktops or laptops. In addition, compared to other connected devices, tablet-type devices are safe, cheap, easy to use, and portable, while the ability to add/remove applications (apps) optimized for use in these devices made them more usable compared to other handled devices. Thus, although the first tablet-type device (iPad) had been offered mainly for business users, it emerged almost immediately as the dominant entertainment medium and learning tool in both formal and informal learning environments [2].
Their unique features described above as according with their relatively low cost made these gadgets, especially tablets, an attractive medium for preschoolers and schoolchildren [3]. This was particularly revolutionary for very young children, even as young as two years old, because touch screens provided new ways of effectively interacting with technology without intermediate tools, such as mice and keyboards, which require a higher level of fine motor skills [4]. The nonprofit organization Common Sense Media in the United States reports that the proportion of children up to eight years of age who have access to a mobile device at home has risen sharply from 52% in 2011 to 75% in 2013 [5]. Given the widespread use of smart devices, it is not surprising that the number of applications developed for them claiming to provide educational experiences to young children has risen accordingly [6,7].
It is documented that, with appropriate usage, content, and context, these apps may indeed contribute positively to learning, as they allow users to personalize their experience, while interactivity features lead to more effective student engagement compared to other machine learning approaches [8,9]. Apps targeted at parents of young children (3–6 years old) dominate the market. Indeed, between 2009 and 2011, apps designed for young children in the Apple Store saw the most growth with a 23% increase, while 58% of the top 100 apps were targeted at children of that age group [10]. Developers categorize their software as educational, claiming that it can help children improve their learning skills. On the other hand, there is broad consensus among researchers that developers are simply taking advantage of a rapidly expanding market by creating a wide variety of applications aimed at a younger audience [11] in order to attain easy and rapid enrichment.
The vast majority of the apps available to download are drill-and-practice style, promoting rote learning. This may be due to the fact that developers or development teams do not follow a specific curriculum and/or often design with little or no input from educators or developmental specialists. In fact, self-proclaimed educational apps simply replace printed textbooks and well-known learning tools such as memory cards [3,4,9]. In this context, research has shown that, although they can indeed transform the learning process and make it easier and interesting in flexible ways, especially by actually adapting to different children’s needs, many of the ‘educational apps’ available are not the best way for very young children to learn math basics, literacy, computational thinking skills, etc. [9].
The question that needs to be considered is whether there is an assessment tool that could be used by parents, custodians, or teachers to help them make the most appropriate decisions. The literature review has shown that although there are several evaluation scales, these are either not copyright-free or lack a sound scientific background. Therefore, there is a crucial demand in the literature for the development of a rapid, simple, and valid assessment tool based on advanced statistical procedures, which allows one to address validity and reliability issues and guide the selection of high-quality educational apps for pre-K children among the thousands of options in mobile app stores.

2. Literature Review

2.1. Educational Apps

While the need of schools, students, educators, and parents for apps that take advantage of the latest mobile and touchscreen technology is high, the majority of educational apps that are available in popular stores such as Google Play and Apple’s App Store, both free and fee-based, have no guarantee of educational value [12]. In the United States, a report from the New America Foundation and the Joan Ganz Cooney Center at Sesame Workshop characterizes the current state of the ‘educational’ app market aimed at young children as a ‘Digital Wild West’, suggesting that parents should be wary of those claims [13].
For parents and educators, choosing an appropriate educational application is a great challenge [14]. The issue of what constitutes an educational app is therefore strikingly complex, since it implies the consideration of various scientific aspects. Thus, sometimes it is easier to identify what constitutes a lack of quality. For instance, Martens, Rinnert & Andersen [15] report that the presence of ads, including pop-ups and pop-unders, poor or inadequate design, and non-functional elements are disruptive to the educational process, while privacy violation issues, etc. further diminish the value of an app.
Kucirkova, Messer, Sheehy & Panadero [16] state that researchers who aim at proposing a conceptual framework for mobile learning applications face many of the same challenges as those researching educational software used for desktop computers. To highlight that, Hirsh-Pasek and her colleagues describe the current app market as the ‘first wave of application development’, in which already-existing non-digital material is being converted into a digital format [17]. Indeed, most of them are found to be reproductions of their print-based counterparts of simple, enjoyable activities offering just passive learning experiences, even though apps with educational value should focus primarily on promoting education, and not just being entertaining [16,18].
Shuler, Levine & Ree [10] analyzed the best children’s educational apps by evaluating the 100 educational apps available for the iPad and iPhone devices (200 apps in total). They found that more than 80% of top-selling paid apps in the Education category target children, 72% of which are designed for preschool-aged children. The study also revealed that developers’ target audience was primarily parents seeking to cultivate a creative environment at home for their children. For anyone that is not a mobile educational technology expert, finding high-quality and appropriate educational apps requires a great deal of time, effort, and luck because this procedure is not only hampered by both the sheer volume available in the stores and the inconvenient digital store user interface, but also by factors such as the lack of description, the misleading scoring system, the subjective user comments, ineffective and unworkable search algorithms, etc. [19]. Martens et al. [15] noted that a simple search in the Apple App Store using the terms ‘A, B, C’ or ‘Alphabet’ returned approximately 279 to 286 results. Indeed, the world’s two major smart device app stores do not provide the users with a user-friendly interface in which navigation is easy and reliable. Moreover, the information included on the principles followed and the methodology used by the development team is often not sufficient for successful decision-making. Although one might argue that information about apps is available in digital stores, this information cannot be used as a general criterion for evaluating the educational value. In fact, this content often comes from the app’s creator, and therefore cannot be considered as accurate or reliable [20].
In addition, there are very few tools for evaluating applications. Although there may be assessment tools in the form of rubrics and checklists developed by researchers at universities, parents and teachers either ignore their existence or find it difficult to use and interpret the results [15]. Researchers such as Hirsh-Pasek et. al and Kucirkova [17,21] also emphasize the fierce competition in the app market. Kucirkova [21] states that developing an application is a costly endeavor; the average cost ranges from 10,000 to 70,000 USD. At the same time, the average fee is about 3 USD, while most Android and Apple apps are available for free download.
Given that the app market is highly competitive with dozens of new products introduced every week, commercial success is not just a result of their quality; it is also a matter of luck. In fact, success relates closely to the number of users who have chosen any given app from a plethora of similar products. Therefore, rapid growth in production and sale is a survival bet for most of the developers That may be a possible explanation about the phenomenon that lots of children’s apps offer the same content with a slightly modified design [17], resulting into lack of effectiveness in academic terms while choosing among the most popular educational apps [17,21]. Indeed, the popularity measured by user reviews, star ratings, or the number of installations is often misleading for parents and teachers, who make a choice based solely on the aforementioned subjective and therefore unreliable criteria [22].

2.2. Are There Tools to Help the General Population to Choose Appropriate Apps?

The low quality of the majority of educational apps targeting preschool-aged children highlights the need for a tool to help parents and educators to evaluate the self-proclaimed educational apps for their real value. In 2019, using the validated PRISMA methodology (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [23], various databases and digital repositories were searched for studies that met the following inclusion criteria: (1) Field study in smart mobile devices, (2) describes an educational app assessment tool, (3) reports on educational apps for children, and (4) is scientifically sound.
The results collected during the review reinforced the importance of evaluation tools for educational apps. The study found 11 articles describing two different assessment approaches. Six studies present a rubric and five studies present a checklist. Additionally, the study also identified seven nonscientific-based tools. Four web sources present a rubric and three sources present a checklist. According to the researcher, the term ‘nonscientific-based tools’ refers to freely available online tools for the evaluation of educational apps that present their quality criteria without a focus on research methodology and scientific evidence.
In general, from all 18 tools, the researcher classified only three [14,24,25] as both comprehensive instruments and scientific-based. The other 15 were characterized as ‘very poor’ in terms of their evaluation power. In general, they were not found to be very comprehensive, thus significantly omitting aspects that are considered important during the evaluation procedure of the educational value of an app. For instance, many tools do not take into consideration the presence of pop-up advertisements, which is a critical characteristic, as researchers claim that their existence distracts users from the learning process [26].
In fact, even though the three tools were considered to be appropriate for app evaluation, they were assumed as not effective in terms of the effort, experience, and time needed by parents and educators for their use. For instance, the rubric in [24] contains eight pages and 24 items, while the rubric in [14] contains four pages and 18 items. The checklist provided by Lee & Kim [25] is two pages long and contains 33 items. These tools need a considerable amount of time and experience by the users. Nobody can be sure that a parent or teacher would like to spend so much time and effort just to evaluate an app. Researchers must, therefore, find a balance between the length, the evaluation power, and the ease of use of a proposed tool.
In conclusion, the digital market is full of apps that are promoted as educational, but they have little or no pedagogical value because they are often made with limited input from educators or developmental specialists. Moreover, the majority of the tools presented in the relevant literature are not available for the parents, caregivers, and educators of young children, as they are stored in copyrighted digital repositories and databases. However, even if they were available, several questions arise in terms of their appropriateness, the time needed to complete an evaluation, etc. On the other hand, the freely available tools are considered as outdated and not appropriate in terms of their depth and scientific evidence. This literature review pointed out the lack of reliable and easy-to-use evaluation tools and highlighted the need for a new, improved one to help everyone who is interested to choose apps with increased educational value. That tool must be easy to use, reliable, short enough, and able to be used as more than a general guideline.

3. Developing and Exploring E.T.E.A.: An Evaluation Tool for Educational Apps

3.1. The Instrument

An initial battery of items was created based on relevant published rubrics, checklists, or questionnaires attempting to evaluate apps. Following the PRISMA method (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), we identified 11 articles describing two different approaches concerning the evaluation of educational apps. Six studies presented a rubric [27,28,29,30,31] and five presented a checklist [32,33,34,35]. We also conducted internet-based searches for gray literature, and we identified seven, four of which presented a rubric [36,37,38,39] and three presented a checklist [40,41,42]. We also found papers that did not include a rubric or a checklist, but they were considered particularly valuable and were also included in the present study [17,43,44,45] along with the others, as they were in the form of guidelines.
Based on theoretical elaborations, certain properties of educational apps comprise the dimensions on which an evaluation could be established. Thus, the underlying factorial structure of a set of items that operationalizes such properties has to be detectable and explicitly expressed via a factor analysis procedure. To achieve this, an Exploratory Factor Analysis (EFA) was applied to a number of items, which were anticipated to conform to a four-factor structure. The four factors were named: Usability, Efficiency, Parental Control, and Security. Note that the EFA procedure stared with a larger number of items, which were excluded from the final solution as they did not conform to the meaningful structure. Though the instrument is targeting parents, custodians, or teachers, we consider that pre-service teachers of preschool education possessing the proper cognitive and affective assets are a suitable sample to work with in order to explore and establish the psychometric properties of the proposed questionnaire under study. The instrument was named the E.T.E.A.—Evaluation Tool for Educational Apps—and it is presented in the Appendix A.

3.2. Participants

The participants (N = 218) were students at the University of Crete, Greece studying at the Department of Preschool Education. They were first-year students and sophomore taking a relevant course in educational technologies during the winter semester of 2019–2020, and they were relatively familiar with the educational apps. First, they downloaded three educational apps for preschool-aged children to smart portable devices, which were in the Greek language and were chosen to be of similar content but of varying quality. The sample consisted only of female students, so it was not possible to investigate any effects of gender.
The study was approved by the University of Crete Institutional Review Board and followed the university guidelines on ethical practice when researching with adults. Anonymity and confidentiality were guaranteed, and informed consent, as well as the right to withdraw, were the ethical principles adhered to. Procedures were followed to ensure the voluntary nature of being involved in the study.

3.3. Exploratory Factor Analysis (EFA)

In the EFA procedure, Principal Components Analysis (PCA) with Varimax rotation was implemented. Initially, a rescaled transformation (Categorical Principal Components Analysis—CATPCA procedure) [46] was applied to the ordinal data so that the resulting scale was suitable for PCA. Bartlett’s test of sphericity (χ2 = 1755.95, p < 0.001) and the Kaiser–Meyer–Olkin index (0.79) indicated adequate variance for factor analysis. The number of factors was decided on the Kaiser criterion, eigenvalue greater than 1, while only items with loadings greater than 0.60 were kept. The four factors explained 79.39% of the total variance. Table 1 shows the corresponding eigenvalues for Usability (4.65), Efficiency (2.79), Parental Control (1.8), and Security (1.08), while the corresponding portions of variance explained were 35.78, 21.45, 13.84, and 8.32, respectively.
The rotated factor structure is shown in Table 2. Reliability analysis of the four dimensions showed that the Cronbach’s coefficient values for Usability, Efficiency, Parental Control, and Security were 0.91, 0.83, 0.96, and 0.73, respectively. Thus, the internal consistency is adequate, and the desirable reliability of the measurements is expected to be satisfactory. Note also that, besides PCA, the principal axis factoring method was applied, which resulted in the same factor structure. The overall factor and reliability analyses indicate and verify the factorial validity of the E.T.E.A. instrument. Table 2 includes the thirteen items allocated to the factors. The four factors are correlated in reality. Table 3 shows the correlation matrix for the four factors. Usability and Efficiency are positively correlated (r = 0.59, p = 0.01). This is a reasonable finding, since an easy-to-use app could become efficient in children’s hands. Efficiency is correlated with Parental Control (r = 0.212, p = 0.01) and with Security (r = 0.252 p = 0.01). The fact that the Parental Control is not correlated with Security indicates that apps might alternatively provide parental control with security options. It is important to note here that the four dimensions validated in this endeavor represent essential properties of educational apps that a kid and a parent anticipate. The usability is a primary aspect to consider for facilitating a kid’s involvement and efficient use of all encoded activities in order to enjoy and learn through them. On the other hand, parents care for security aspects, and it is reasonable to demand information about what their children do.

4. Limitations

In addition to the importance and striking empirical findings, this study has some limitations. Since it is the first attempt to validate these assessment aspects of apps, the findings should be replicated with wider samples, larger numbers of apps, and varied populations, e.g., parents and/or educators, in order to establish the instrument’s validation and its generalized psychometric properties. Moreover, the four-factor structure should not be considered as complete, but it could be further extended to include additional dimensions dictated by other theoretical premises that we might foster.

5. Discussion and Conclusions

The present work contributes toward the development of a valid instrument for assessing educational apps for children aged 3–6 years. As was mentioned earlier, the EFA refinement started with a larger number of items, which were excluded from the final solution as they did not conform to the meaningful structure. The final proposed dimensions were: Usability, Efficiency, Parental Control, and Security. These dimensions were conceived in an initial theoretical elaboration, taking into consideration the relevant functions of the apps, their purposes, children’s expectations, and surely the parents’ implicit demands. The resulting four-dimensional structure satisfies the validity and reliability presuppositions for a scientifically developed instrument. It is important, however, to emphasize here that this factor structure, even though valid, is not complete, as additional dimensions (e.g., related to learning experiences and outcomes) could be found to exist and be incorporated into the present structure. Although some apps can provide a rich learning experience to enhance young children’s knowledge while maintaining a high level of satisfaction in both formal and informal learning environments, the majority of self-proclaimed ‘educational’ apps simply do not provide significant evidence to substantiate that title [47]. On the other hand, as far as the measurement is concerned, learning outcomes are a difficult case, and merely reporting one’s personal opinions does not guarantee valid judgment. Assessing learning demands more objective approaches and a significantly more complicated procedure. In contrast, the easy-to-use aspects are more obvious and are straightforwardly evaluated. Indirectly, these could be considered as facilitating learning outcomes. A relatively small number of apps are well designed, easily navigable, and offer innovative approaches to support children to learn more effortlessly and effectively. In their majority, app developers build their apps without having an understanding of how children learn and develop [12]. As a result, they do not create digital environments that foster children’s engagement in play activities that promote learning across academic and social domains. They reproduce printed learning tools and material in a digital format that promotes the acquisition of knowledge or skill through repetitive practice [1,14]. Efficiency is the other mediator towards learning; it is considered essential and is easier to evaluate. Since some apps are more effective than others in facilitating a learning-by-doing approach and improving levels of children’s engagement in the learning process, a question that arises is how teachers and parents recognize them among others. Teachers and parents have to spend many hours trying to find quality learning apps for children to support their learning experiences both at school and at home [48].
So, what can teachers and parents do when looking for ‘an educational app’? They need to be able to assess its suitability and determine whether it is appropriate based on the child’s age or level of development [1]. Vaala, Ly & Levine [12] claim that it is essential to go beyond the descriptions and comments in the app stores before downloading an app, and also to visit the developers’ websites. Additionally, Callaghan [9] suggests that parents must play an active role in their children’s learning by playing with apps with their children.
No one can deny that smart mobile devices and their accompanying apps are here to stay, and they have already transformed the way that young children experience and act in the world [49]. Therefore, it is necessary for those involved in the application of technology within the formal and informal learning environments to focus on improving educational outcomes by creating more efficient apps that enhance the cognitive development of young children.
Until that happens, however, it is essential for parents and/or educators to have a reliable, powerful, fast, and easy-to-use tool that will guide them in choosing apps with actual educational value for their children and/or students. The tool presented in this paper also aims to contribute in this direction. Given the multidimensional underlying structure of this evaluation tool, further exploration, and extension of the E.T.E.A. is possible and is certainly needed to attain a more complete instrument, a valued asset of mainly parental concern.

Author Contributions

All co-authors contributed to data collection and/or analysis of project results. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

We would like to thank the participant students for their contribution in our investigation of the E.T.E.A. tool.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Evaluation Tool for Educational Apps
The eleven items comprising the E.T.E.A./questionnaire are presented below:
Evaluation instructions
Please respond by marking the extent to which you consider each of the following statements to be applicable to the app that you evaluate. Please place your response on the scale ranging from 1 (strongly disagree) to 5 (strongly agree). Keep in mind that there are no right or wrong answers!
Table
Table A1. Appraise the properties of the App.
Table A1. Appraise the properties of the App.
Q1—App’s instructions are presented in a way that is appropriate for children.
Q2—App’s visual elements follow the same order throughout the entire app.
Q3—App’s elements (menus, buttons, navigation arrows) are properly positioned to facilitate use.
Q4—App can be easily used by children.
Q5—App offers parametrization options.
Q6—App offers multimedia options suitable for children.
Q7—App offer multiple ways for conveying information (through images, sounds, words, speech).
Q8—App provides clear feedback to children about each activity.
Q9—App provides consultation with parents.
Q10—App informs the parents about the child’s progress (e.g., via email).
Q11—App does not urge my child to do any type of online transaction.
Q12—App does not contain destructive ads (pop-up messages).
Q13—App explicitly states the personal data management policy.

References

  1. Papadakis, S.; Zaranis, N.; Kalogiannakis, M. Parental involvement and attitudes towards young Greek children’s mobile usage. Int. J. Child Comput. Interact. 2019, 22, 100144. [Google Scholar] [CrossRef]
  2. Kalogiannakis, M.; Nirgianaki, G.M.; Papadakis, S. Teaching magnetism to preschool children: The effectiveness of picture story reading. Early Child. Educ. J. 2018, 46, 535–546. [Google Scholar] [CrossRef]
  3. Kabali, H.K.; Irigoyen, M.M.; Nunez-Davis, R.; Budacki, J.G.; Mohanty, S.H.; Leister, K.P.; Bonner, R.L. Exposure and use of mobile media devices by young children. Pediatrics 2015, 136, 1044–1050. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Wohlwend, K.E. Who gets to play? Access, popular media and participatory literacies. Early Years 2017, 37, 62–76. [Google Scholar] [CrossRef]
  5. Common Sense Media. Zero to Eight: Children’s Media Use in America 2013. Common Sense Media 2013, Research Study. Available online: https://www.commonsensemedia.org/research/zero-to-eight-childrens-media-use-in-america-2013 (accessed on 7 April 2020).
  6. Godfrey, B.J.; Reed, M. App Store after Five Years. 2013. Available online: https://actonline.org/wp-content/uploads/2014/04/The-App-Store-After-Five-Years.pdf (accessed on 7 April 2020).
  7. Papadakis, S.; Kalogiannakis, M.; Zaranis, N. The effectiveness of computer and tablet assisted intervention in early childhood students’ understanding of numbers. An empirical study conducted in Greece. Educ. Inf. Technol. 2018, 23, 1849–1871. [Google Scholar] [CrossRef]
  8. Calder, N. Apps: Appropriate, applicable, and appealing? In Digital Games and Mathematics Learning, Mathematics Education in the Digital Era 4; Lowrie, T., Jorgensen, R., Eds.; Springer: Zevenbergen, The Netherlands, 2015; pp. 233–250. [Google Scholar]
  9. Callaghan, M. Educational Apps Designed for Kids: Avoiding Missed Opportunities. 2018. Available online: https://bold.expert/educational-apps-designed-for-kids-avoiding-missed-opportunities/ (accessed on 7 April 2020).
  10. Shuler, C.; Levine, Z.; Ree, J. iLearn II an Analysis of the Education Category of Apple’s App Store. 2012. Available online: https://joanganzcooneycenter.org/publication/ilearn-ii-an-analysis-of-the-education-category-on-apples-app-store/ (accessed on 7 April 2020).
  11. Papadakis, S.; Kalogiannakis, M. Evaluating a course for teaching introductory programming with Scratch to pre-service kindergarten teachers. Int. J. Technol. Enhanc. Learn. 2019, 11, 231–246. [Google Scholar] [CrossRef]
  12. Vaala, S.; Ly, A.; Levine, M.H. Getting a read on the app stores: A market scan and analysis of children’s literacy apps. Full report. In Joan Ganz Cooney Center at Sesame Workshop; Joan Ganz Cooney Center at Sesame Workshop: 1900 Broadway, New York, NY, USA, 2015. [Google Scholar]
  13. Guernsey, L.; Levine, M.H. Pioneering literacy in the digital age. Technol. Digit. Media Early Years Tools Teach. Learn. 2015, 104–114. [Google Scholar]
  14. Papadakis, S.; Kalogiannakis, M.; Zaranis, N. Educational apps from the Android Google Play for Greek preschoolers: A systematic review. Comput. Educ. 2018, 116, 139–160. [Google Scholar]
  15. Martens, M.; Rinnert, G.C.; Andersen, C. Child-centered design: Developing an inclusive letter writing app. Front. Psychol. 2018, 9, 2277. [Google Scholar] [CrossRef]
  16. Kucirkova, N.; Messer, D.; Sheehy, K.; Panadero, C.F. Children’s engagement with educational iPad apps: Insights from a Spanish classroom. Comput. Educ. 2014, 71, 175–184. [Google Scholar] [CrossRef]
  17. Hirsh-Pasek, K.; Zosh, J.M.; Golinkoff, R.M.; Gray, J.H.; Robb, M.B.; Kaufman, J. Putting education in “educational” apps: Lessons from the science of learning. Psychol. Sci. Public Interest 2015, 16, 3–34. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Lynch, J.; Redpath, T. ‘Smart technologies in early years literacy education: A meta-narrative of paradigmatic tensions in iPad use in an Australian preparatory classroom. J. Early Child. Lit. 2014, 14, 147–174. [Google Scholar] [CrossRef]
  19. Larkin, K. An app! An app! My kingdom for an app: An 18 month quest to determine whether apps support mathematical knowledge building. In Digital Games and Mathematics Learning: Potential, Promises and Pitfalls; Lowrie, T., Jorgensen, R., Eds.; Springer: Dordrecht, The Netherlands, 2015; pp. 251–276. [Google Scholar]
  20. Larkin, K. Mathematics education: Is there an app for that? In Proceedings of the 36th Annual conference of the Mathematics Education Research Group of Australasia. Mathematics education: Yesterday, Today and Tomorrow 2013, Melbourne, VI, Australia, 7–11 July 2013; Steinle, V., Ball, L., Bardini, C., Eds.; MERGA: Melbourne, Australia, 2013; pp. 426–433. [Google Scholar]
  21. Kucirkova, N. Trans-and intra-apps: Innovating the app market and use. In Apps, Technology and Younger Learners. In Apps, Technology and Younger Learners: International Evidence for Teaching 2016; Falloon, G., Kucirkova, N., Eds.; Routledge: New York, NY, USA; London, UK, 2016; pp. 272–284. [Google Scholar]
  22. Flewitt, R.; Kucirkova, N.; Messer, D. Touching the virtual, touching the real: iPads and enabling literacy for students experiencing disability. Aust. J. Lang. Lit. 2014, 37, 107–116. [Google Scholar]
  23. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann. Intern. Med. 2009, 151, 264–269. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Lee, C.-Y.; Cherner, T.S. A comprehensive evaluation rubric for assessing instructional apps. J. Inf. Technol. Educ. Res. 2015, 14, 21–53. [Google Scholar] [CrossRef] [Green Version]
  25. Lee, J.S.; Kim, S.W. Validation of a tool evaluating educational apps for smart education. J. Educ. Comput. Res. 2015, 52, 435–450. [Google Scholar] [CrossRef]
  26. Dorouka, P.; Papadakis, S.; Kalogiannakis, M. Tablets & apps for promoting Robotics, Mathematics, STEM Education and Literacy in Early Childhood Education. Int. J. Mob. Learn. Organ. 2020, 14, 255–274. [Google Scholar]
  27. Walker, H.C. Establishing content validity of an evaluation rubric for mobile technology applications utilizing the Delphi method. Doctoral Thesis, Johns Hopkins University, Baltimore, MD, USA, 2013. [Google Scholar]
  28. Green, L.S.; Hechter, R.P.; Tysinger, P.D.; Chassereau, K.D. Mobile app selection for 5th through 12th grade science: The development of the MASS rubric. Comput. Educ. 2014, 75, 65–71. [Google Scholar] [CrossRef]
  29. Israelson, M.H. The app map: A tool for systematic evaluation of apps for early literacy learning. Read. Teach. 2015, 69, 339–349. [Google Scholar] [CrossRef]
  30. Ok, M.W.; Kim, M.K.; Kang, E.Y.; Bryant, B.R. How to find good apps: An evaluation rubric for instructional apps for teaching students with learning disabilities. Interv. School Clin. 2016, 51, 244–252. [Google Scholar] [CrossRef] [Green Version]
  31. Papadakis, S.; Kalogiannakis, M.; Zaranis, N. Designing and creating an educational app rubric for preschool teachers. Educ. Inf. Technol. 2017, 22, 3147–3165. [Google Scholar] [CrossRef]
  32. McManis, L.D.; Gunnewig, S.B. Finding the education in educational technology with early learners. Young Child. 2012, 67, 14–24. [Google Scholar]
  33. More, C.M.; Travers, J.C. What’s app with that? Selecting educational apps for young children with disabilities. Young Except. Child. 2013, 16, 15–32. [Google Scholar] [CrossRef]
  34. Weng, P.L. Developing an app evaluation rubric for practitioners in special education. J. Spec. Educ. Technol. 2015, 30, 43–58. [Google Scholar] [CrossRef]
  35. Lubniewski, K.L.; Arthur, C.L.; Harriott, W. Evaluating instructional apps using the app checklist for educators (ACE). Int. Electron. J. Elem. Educ. 2018, 10, 323–329. [Google Scholar] [CrossRef] [Green Version]
  36. Van Houten, J. Ievaluate app Rubric. 2011. Available online: https://static.squarespace.com/static/50eca855e4b0939ae8bb12d9/50ecb58ee4b0b16f176a9e7d/50ecb593e4b0b16f176aa97b/1330388174777/JeanetteVanHoutenRubric.pdf (accessed on 23 April 2020).
  37. Vincent, T. Ways to Evaluate Educational Apps. 2012. Available online: http://learninginhand.com/blog/ways-to-evaluate-educational-apps.html (accessed on 23 April 2020).
  38. Walker, H. Evaluation Rubric for iPod Apps. 2010. Available online: http://learninginhand.com/blog/evaluation-rubric-for-educational-apps.html (accessed on 23 April 2020).
  39. Schrock, K. Evaluation Rubric for iPod/iPad Apps. 2011. Available online: https://www.ipads4teaching.net/uploads/3/9/2/2/392267/ipad_app_rubric.pdf (accessed on 23 April 2020).
  40. Schrock, K. Critical Evaluation of a Content-Based iPad/iPod App. 2015. Available online: https://www.ipads4teaching.net/uploads/3/9/2/2/392267/evalipad_content.pdf (accessed on 23 April 2020).
  41. KIDMAP. The Dig Checklist for Inclusive, High-Quality Children’s Media. 2017. Available online: https://www.joinkidmap.org/digchecklist/ (accessed on 23 April 2020).
  42. Haines, C. Evaluating Apps and New Media for Young Children: A Rubric. 2016. Available online: https://nevershushed.files.wordpress.com/2016/09/2016evaluatingappsandnewmediaforyoungchildrenarubric.pdf (accessed on 23 April 2020).
  43. Marsh, J.; Plowman, L.; Yamada-Rice, D.; Bishop, J.; Lahmar, J.; Scott, F. Play and creativity in young children’s use of apps. Br. J. Educ. Technol. 2018, 49, 870–882. [Google Scholar] [CrossRef] [Green Version]
  44. Rosell-Aguilar, F. State of the app: A taxonomy and framework for evaluating language learning mobile applications. CALICO J. 2017, 34, 243–258. [Google Scholar] [CrossRef]
  45. Zosh, J.M.; Lytle, S.R.; Golinkoff, R.M.; Hirsh-Pasek, K. Putting the education back in educational apps: How content and context interact to promote learning. In Media Exposure during Infancy and Early Childhood 2017; Barr, R., Linebarger, D.N., Eds.; Springer: Cham, Germany, 2017; pp. 259–282. [Google Scholar]
  46. Meulman, J.J.; Van Der Kooij, A.J.; Heiser, W.J. Principal components analysis with nonlinear optimal scaling transformations for ordinal and nominal data. In The Sage Handbook of Quantitative Methodology for the Social Sciences; Kaplan, D., Ed.; Sage: Thousand Oaks, CA, USA, 2004; Volume 3, pp. 49–70. [Google Scholar]
  47. Hiniker, A. Research Suggests Well-Designed Preschool Apps Can Encourage Family Engagement and Self-Regulation. 2018. Available online: http://joanganzcooneycenter.org/2018/05/21/research-suggests-well-designed-preschool-apps-can-encourage-family-engagement-and-self-regulation/ (accessed on 7 April 2020).
  48. Moffett, P.; Gray, C.; Dunn, J.; Mitchell, D. Benefits of mobile digital devices in early learning. Early Years Educ. 2017, 19, 26–28. [Google Scholar] [CrossRef]
  49. Falloon, G. Young students using iPads: App design and content influences on their learning pathways. Comput. Educ. 2013, 68, 505–521. [Google Scholar] [CrossRef]
Table
Table 1. Eigenvalues and percent variance explained by the four factors obtained by Principal Components Analysis (PCA).
Table 1. Eigenvalues and percent variance explained by the four factors obtained by Principal Components Analysis (PCA).
FactorsEigenvalues% of VarianceCumulative %
1Usability4.6535.7835.78
2Efficiency2.7921.4557.23
3Parental Control1.8013.8471.07
4Security1.088.3279.39
Table
Table 2. Principal Components Analysis with Varimax rotation.
Table 2. Principal Components Analysis with Varimax rotation.
Factors
1234
Q1—Suitable Instructions0.898
Q2—Button array is consistent 0.880
Q3—Buttons facilitate use0.858
Q4—Can be used easily0.853
Q5—Parametrization options 0.859
Q6—Suitable multimedia options 0.722
Q7—Multiple way for conveying information 0.669
Q8—Feedback interaction 0.651
Q9—Consulting parents 0.952
Q10—Inform parents 0.949
Q11—No urging for purchase 0.893
Q12—No destructive ads 0.852
Q13—Inform about personal data policy 0.684
Cronbach’s alpha of internal consistency0.910.830.960.73
Table
Table 3. Correlation Matrix of the four dimensions.
Table 3. Correlation Matrix of the four dimensions.
1234
Usability10.590**0.0900.055
Efficiency0.590 **10.212 **0.252 **
Parental Control0.0900.212**10.001
Security0.0550.252**0.0011
** Correlation is significant at the 0.01 level (two-tailed).

Share and Cite

MDPI and ACS Style

Papadakis, S.; Vaiopoulou, J.; Kalogiannakis, M.; Stamovlasis, D. Developing and Exploring an Evaluation Tool for Educational Apps (E.T.E.A.) Targeting Kindergarten Children. Sustainability 2020, 12, 4201. https://doi.org/10.3390/su12104201

AMA Style

Papadakis S, Vaiopoulou J, Kalogiannakis M, Stamovlasis D. Developing and Exploring an Evaluation Tool for Educational Apps (E.T.E.A.) Targeting Kindergarten Children. Sustainability. 2020; 12(10):4201. https://doi.org/10.3390/su12104201

Chicago/Turabian Style

Papadakis, Stamatios, Julie Vaiopoulou, Michail Kalogiannakis, and Dimitrios Stamovlasis. 2020. "Developing and Exploring an Evaluation Tool for Educational Apps (E.T.E.A.) Targeting Kindergarten Children" Sustainability 12, no. 10: 4201. https://doi.org/10.3390/su12104201

APA Style

Papadakis, S., Vaiopoulou, J., Kalogiannakis, M., & Stamovlasis, D. (2020). Developing and Exploring an Evaluation Tool for Educational Apps (E.T.E.A.) Targeting Kindergarten Children. Sustainability, 12(10), 4201. https://doi.org/10.3390/su12104201

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop