Practitioner Perspectives on COVID-19’s Impact on Computer Science Education Among High Schools Serving Students from Lower and Higher Income Families
Authors: 
Monica McGill, 
Eric Snow, Luronne Vaval, 
Leigh Ann DeLyser, Stephanie Wortel-London, Angelica ThompsonAuthors Info & Claims
Article No.: 12, Pages 1 - 31
Abstract
Research Problem. Computer science (CS) education researchers conducting studies that target high school students have likely seen their studies impacted by COVID-19. Interpreting research findings impacted by COVID-19 presents unique challenges that will require a deeper understanding as to how the pandemic has affected underserved and underrepresented students studying or unable to study computing.
Research Question. Our research question for this study was: In what ways has the high school computer science educational ecosystem for students been impacted by COVID-19, particularly when comparing schools based on relative socioeconomic status of a majority of students?
Methodology. We used an exploratory sequential mixed methods study to understand the types of impacts high school CS educators have seen in their practice over the past year using the CAPE theoretical dissaggregation framework to measure schools’ Capacity to offer CS, student Access to CS education, student Participation in CS, and Experiences of students taking CS.
Data Collection Procedure. We developed an instrument to collect qualitative data from open-ended questions, then collected data from CS high school educators (n = 21) and coded them across CAPE. We used the codes to create a quantitative instrument. We collected data from a wider set of CS high school educators (n = 185), analyzed the data, and considered how these findings shape research conducted over the last year.
Findings. Overall, practitioner perspectives revealed that capacity for CS Funding, Policy & Curriculum in both types of schools grew during the pandemic, while the capacity to offer physical and human resources decreased. While access to extracurricular activities decreased, there was still a significant increase in the number of CS courses offered. Fewer girls took CS courses and attendance decreased. Student learning and engagement in CS courses were significantly impacted, while other noncognitive factors like interest in CS and relevance of technology saw increases.
Practitioner perspectives also indicated that schools serving students from lower-income families had (1) a greater decrease in the number of students who received information about CS/CTE pathways; (2) a greater decrease in the number of girls enrolled in CS classes; (3) a greater decrease in the number of students receiving college credit for dual-credit CS courses; (4) a greater decrease in student attendance; and (5) a greater decrease in the number of students interested in taking additional CS courses. On the flip-side, schools serving students from higher income families had significantly higher increases in the number of students interested in taking additional CS courses.
1 Introduction
During the past 12 months, the education research community has considered many questions about how the COVID-19 pandemic might be impacting their ongoing and planned research studies, including its impact on the education ecosystem. These questions are like many questions asked by researchers when school or participants in their studies fell victim to other natural or man-made disasters [1, 20, 30, 32] or the H1N1 pandemic disruption [5], albeit at a smaller scale. Unpacking these impacts requires understanding the various aspects of the program/study that could potentially be impacted and factoring those impacts in when interpreting any usable data.
Early studies on the impact of COVID-19 have shown a disparity among schools, teachers, and students across the world [15, 36, 38]. Underserved and underrepresented students, from those who are from low-income families to those who are living in rural areas with limited Internet access, have been unwillingly placed in a further disparate educational system than their counterparts [27]. Yet, even their counterparts have been impacted by this unprecedented educational disruption.
As educational communities have started to rise to address inequities in computer science (CS) education, as researchers, we faced a limitation in understanding how the pandemic has impacted progress. Further, we were keen to understand how the pandemic impacted the capacity for schools to offer CS instruction (e.g., courses and extracurricular activities), student awareness of the instruction, and student willingness to enroll in the courses or participate in extracurricular activities. We also needed to know how learning CS remotely during a pandemic impacted their ability to learn about, their interest in, and their perseverance in studying CS–all of which are important factors in students’ academic achievement. We reflected on our ongoing high school CS education research studies and what this significant educational disruption means when interpreting the data. It became apparent that it was necessary to investigate the impact of COVID-19 on CS education in high schools. In particular, the primary research question we sought to answer was: In what ways has the high school computer science educational ecosystem for students been impacted by COVID-19, particularly when comparing schools based on relative socioeconomic status of a majority of students?
Since the landscape of possible impacts of COVID-19 on CS education is broad, we chose to use the Capacity, Access, Participation, and Experience (CAPE) framework [16] to structure our design of a quantitative instrument. CAPE is a theoretical framework for explicitly disaggregating the capacity of institutions, access students have to courses, participation of different student groups, and experience of those students enrolled in CS. We also used the CAPE framework to frame results of the multiple facets of impact on CS education.
This research is important to advocates, decision-makers, and funding bodies that are working to increase the capacity of equitable CS high school education and to researchers who study the same. Though particular to the U.S., other countries may find this work relevant to understanding the impact of COVID-19 on their secondary students. CS education researchers conducting studies that target high school students have likely seen their studies impacted by COVID-19. Interpreting research findings impacted by COVID-19 presents unique challenges that will require a deeper understanding of how the pandemic has affected underserved students’ ability to study computing.
2 Background
Although the last global pandemic was over 100 years ago, other localized events have significantly disrupted education. There have been many studies focused on educational disruption due to natural disasters (e.g., hurricanes, tornadoes, war) [34, 35]. Research into educational disruption has focused on the capacity of educational institutions, access of resources to provide adequate education to students, and interruption and delay of learning for students who were impacted [12]. In this section, we describe some of the emerging literature on K-12 education during COVID-19, implications for CS education especially with regards to equity, and a framework for evaluating the multifaceted impacts of COVID-19 on CS education specifically over the last year. Although we frame the larger problem as one of equity and cite work that includes racial disparities and inequitable access to resources based on the relative wealth of school populations [33], we note that these challenges are often correlated. As the literature at this time is limited due to the emerging nature of the research and recent emergence of global events, we specifically address this in the limitations section and use imperfect theoretical alignment to hypothesize about what types of impacts would be more relevant to perceive.
2.1 Impacts of COVID-19 on K-12 Education
The impacts of the COVID-19 pandemic on primary and secondary education, while still evolving and emerging, have been profound and widespread [15, 17, 38, 39]. Much of the available research indicates that the pandemic has magnified existing inequities in K-12 education systems across the globe [13, 49], impacting school capacity to offer adequate training and resources for teachers to move instruction partially or fully online, as well as student access to help, technology and stable, supportive learning environments. In turn, these have impacted student participation in online and hybrid courses and, ultimately, their engagement and academic performance. The overall implications of these impacts are hard to ignore, with one study [38] indicating that the current academic cohort could lose up to one-third of a year of instructional time, a loss that will undoubtedly be exacerbated for underserved student populations [13, 14]. Subject-specific learning losses are still being investigated, but some evidence [14, 23, 37, 40] suggests that math and possibly other STEM subject-areas may see the brunt of these losses.
Participation has been a key issue for students. Onyema et al. examined the impacts of COVID on their education system and found that many educators and students relied on technology to continue learning online, but that online education was negatively impacted by factors such as poor infrastructure, technology inaccessibility and unavailability issues, and poor digital skills [36]. In Pakistan, Adnan and Anwar found something similar–that COVID-induced online learning faced significant challenges to success to the vast majority of students who were unable to access the Internet due to technical and monetary issues [2]. Public media in the U.S. has echoed similar observed challenges [46].
In the U.S., the Institute of Education Sciences (IES) conducted a national survey monthly from January through May 2021 to provide insights into learning opportunities offered by schools during the COVID-19 pandemic [21]. Results from February 2021 offer evidence of enrollment trends across different groups for fourth and eighth grade student cohorts. Some of the key trends included:
•
Nationally, Black and Hispanic students in 4th and 8th grades were enrolled in fully remote learning models at higher rates (55–60% versus the average 42–45%), and at much higher rates than 4th and 8th grade White students (24–27%). The percentages of 4th and 8th grade Black and Hispanic students enrolled in fully remote learning models (55–60%) was similar to both the percentage of economically disadvantaged 4th and 8th grade students enrolled in fully remote learning models (48–52%), and the percentage of ELL 4th and 8th grade students enrolled in fully remote learning models (53%–58%).
•
With respect to regional differences, it is worth noting that in almost every category the Western U.S. had significantly higher percentages of both 4th and 8th grade students enrolled in fully remote learning models than the other three US regions (Northeast, South, Midwest).
•
With respect to school location differences, it is worth noting the discrepancy in primary learning models between Urban/Suburban and Town/Rural locations. Most 4th and 8th grade students in Urban/Suburban locations were enrolled in fully remote learning models, whereas most students in Town/Rural locations were enrolled in full in-person learning models. This trend was even more pronounced for Black, Hispanic, Economically Disadvantaged ELL students, but less pronounced for White students.
This evidence indicates some key differences in enrollment trends across remote, hybrid and in-person learning models for key student groups. What is still largely unknown at this point is the extent to which participation in different learning models impacts learning outcomes. There is early evidence that when students have had to move learning online their eventual success is mediated by their access to technology, academic and peer support, and stable learning environments. We also know from past evidence that Black, Hispanic, ELL, and Economically Disadvantaged students are more likely to experience challenges with access to technology, support and stable learning environments, making their eventual success in online learning environments less likely.
Coping with these changes has not been easy for students. Cao et al. examined the impacts of the pandemic on college students and found that just over 20% reported experiencing mild to moderate anxiety, an outcome that was positively mediated by living in urban areas, family income stability, and living with parents [3]. The results also indicated that the economic effects, effects on daily life, and delays in academic activities were positively associated with anxiety symptoms. Anxiety is known to negatively affect emotional engagement of students in learning [25], which can further exacerbate the negative impact of learning during the pandemic.
2.2 Early Impacts of COVID-19 on CS Education
The COVID-19 pandemic has also impacted CS education worldwide. In the U.S., the CS Teachers Association (CSTA) and the Kapor Center [27] found that fewer than 20% of teachers reported temporarily suspending CS instruction, but higher rates were reported by teachers at schools serving rural, low-income, and Black, Latinx, and Indigenous students. Further, just over 40% of teachers reported distance learning as a major challenge to instruction, with teachers at schools serving rural, low-income and Black, Latinx, and Indigenous students more likely to face these challenges. In the U.K., Mooney and Becker investigated the impacts of the pandemic on CS students’ sense of belongingness and how it changed as they switched to online learning [31]. They found statistically significant reductions in the belongingness of students identifying as men, as well as those not identifying as being part of a minority, but more nuanced trends among students with self-identified gender and minority status.
Three other US studies investigated the impacts of having to move CS education online from different angles. Focusing on middle and high school students participating in a Girls Who Code camp, McDonald and Dillon found that despite a significant decrease in attendance when it was moved online, those who did participate reported high morale and satisfaction [28]. Skuratowicz et al. studied virtual summer PD to help elementary and bilingual teachers learn the fundamentals of computational thinking [41]. They found that it was important to proactively address elementary teachers’ barriers to technology adoption and ask teachers to include the physical in the virtual whenever possible to keep the focus on hands-on learning. In another study, Computing Research Association found that faculty reported the greatest challenges for their students moving online were family obligations, insufficient or slow internet, and mental health issues [8]. Meanwhile, faculty reported that implementing their desired instructional style virtually was challenging as was the amount of time it took to teach online.
Crick et al. looked at the impacts of COVID-induced emergency remote teaching on the CS education community [10]. Their findings indicated that those who work within the CS discipline were significantly more likely than educators in other disciplines to say that they felt prepared, confident, supported by their institution, held a good working knowledge of appropriate technologies, had access to appropriate technologies, and were confident that their students could access online lessons and assessments.
2.3 Equity in K-12 CS Education
CS education, especially in the United States, has focused on the challenges of broadening participation, access, and initiating new programs across the landscape. The CS for All movement and the larger research community, often use data to show a deficit narrative, make visible the negative space of implementation, and close gaps of access and participation in communities [43]. More recently, there is emerging sophistication of dialog regarding the difference between broadening participation and equity [50]. Over the last decade, K-12 CS education has reformed its measures to not just identify growth in numbers of student participation, but truly ask questions about who is missing, and for students who are participating if they are succeeding at similar rates to their White and Asian peers [43].
CS education in K-12 is entering a space already fraught with inequitable access, participation, and educational experiences and outcomes for students. The disaggregated lens of not just student participation but the full ecosystem of factors impacting the ability for access, participation, and experience to have comparable outcomes across student subgroups is an important and ongoing conversation in K-12 education in the United States [26]. Although international communities may not have to contend with the same structural bias and racism that persists in the United States due to more centralized control of education systems, the lessons learned here can be important to identifying hidden factors in how COVID-19 may have affected communities differently across the globe, as many of the same differences in outcomes (especially the participation of women) persist in international settings.
In this study, we use the CAPE framework to specifically disaggregate the impacts of COVID-19 on various components of students’ CS education experience. This allows for clarity in the observations for problems of capacity, access, participation, and experience that help identify institutional barriers compared to individual barriers when systems are stressed. We again remind the reader that our work specifically disaggregates the types of schools by socioeconomic status of the majority of students [33], as measured by free and reduced lunch, but this measure often strongly correlates with rural schools or schools that serve a high population of BIPOC students.
2.4 The CAPE Framework
It is clear from this review that COVID has disrupted many aspects of education systems worldwide. In particular, the massive and sudden shift to online learning has exacerbated many equity-related education issues, including capacity of schools to offer CS courses and extracurricular activities, access to technology, stable and supportive home learning environments, students’ sense of belongingness and anxiety, and students’ learning experiences. CS education has not been spared these impacts, with several studies echoing challenges in other subject areas. It is essential then, when seeking a comprehensive understanding of the impacts of COVID on education and CS education, that a systems-level framework and approach is adopted so that evidence can be developed at multiple levels. Fletcher and Warner developed the CAPE framework to help education leaders, policymakers, and researchers evaluate equity in CS education at multiple levels of educational systems [16]. The framework (seen in Figure 1) takes a systems-level approach that compels us to consider student outcomes and how those outcomes are situated within a larger initiative and policy level environment.1 This illustrates how the framework components are interrelated and rely upon previous components.
Fig. 1.
Students are more likely to have positive CS learning experiences when they elect to participate in CS courses and programs, and they choose to participate in CS when they have access to CS courses and programs. Schools can provide students access to CS courses and programs when they have the capacity to offer CS courses and programs. Using CAPE as a disaggregation model of findings, the New York University (NYU) Research Alliance has used CAPE to investigate the New York City (NYC) CS4ALL initiative. Although NYC’s initiative is a single district, the scale of the initiative makes it unique - in 2020, there were 160,000 students in over 1,200 schools enrolled in initiative-based CS coursework [52], giving indication that the framework is robust and useful.
3 Research Plan
To answer the research question, In what ways has the high school computer science educational ecosystem for students been impacted by COVID-19, particularly when comparing schools based on relative socioeconomic status of a majority of students?, we planned to conduct an explanatory sequential mixed methods design [9, 22], with the first part of the study focusing on the quantitative data collection followed by a qualitative study that offered explanations to the quantitative study’s results. For this study, we adapted the CAPE framework to ensure appropriate measures at multiple levels connecting student participation and experiences to district and school contexts where access and capacity are key factors (Figure 2). Here, we provide context for each component:
Fig. 2.
•
Capacity: A district’s or school’s ability to offer equity-focused policies, resources, and funding. This includes the extent to which school leadership, staff, and teachers are effectively prepared to implement equity-focused CS courses, advising, and extracurricular activities.
•
Access: Students’ equitable access to CS courses, advising, and extracurricular activities.
•
Participation: Students’ awareness of and enrollment in CS courses, extracurricular activities. This includes the extent to which they enroll in them in equal proportions.
•
Experience: Equitable student outcomes in CS courses and engagement in CS-focused college and career options. This includes the extent to which the course is equally and positively impacting cognitive and noncognitive outcomes, including interest in attending college and awareness of career options.
In studies like this, there is typically a basis for collecting the quantitative data, such as existing instrumentation or previous research that can be referenced for creating new instrumentation. However, there is limited research on the impact of COVID-19 (or similar) on education in general and, at the beginning of this study, no previous research on the impact of COVID-19 on CS education in high schools. That is, we had limited background research to build upon and no existing instrumentation. Further, since the CAPE framework is also relatively new with limited published instrumentation on measuring the various components, we chose to modify the research design (quantitative followed qualitative) as follows (Figure 3):
Fig. 3.
•
Create an instrument to collect qualitative data from open-ended questions deeply rooted in the four CAPE components. Once created, collect data from CS high school educators.
•
Using the qualitative data, analyze the responses, code them across CAPE, and then cultivate them as more refined questions for integration into a second instrument. Once created, collect data from a wider set of CS high school educators.
•
Using the results from above, create a semi-structured interview protocol for qualitative data collection and interview 6 to 8 high school educators.
Steps 1 and 2 comprise the modified exploratory sequential mixed methods study [29] and are the focus of this paper. Step 3 will conclude the explanatory sequential mixed-methods study portion of the overall study, planned for later this year. All three parts of this study were approved as exempt human subjects research by an Institutional Review Board.
3.1 Researcher Positionality Statement
Though positionality statements are not often seen as common practice in quantitative studies, we believe they should be. As researchers involved in this study, we bring various interpretations of both the qualitative and quantitative data based on our experiences with conducting research, designing and implementing qualitative, quantitative, and mixed-methods studies, and promoting equitable CS education locally, regionally, and nationally within the United States (and also internationally). With equity-focused initiatives, a large part of these programs and initiatives across the educational ecosystem guided us in our creation of instruments, collecting data, and interpreting the results.
Members of our team have extensive experience studying inequities in computer science education, from the systems-level approach to impact on learning on various population subgroups (e.g., girls/women learning CS, race/ethnicity). Several of the team members have over 15 years experience conducting this type of research. We bring our unique blend of experience and understanding of the education ecosystem (including two have taught CS in high school and college) to this study.
We are firm believers that that the COVID-19 pandemic had a serious impact on all of our lives, and the shift to online communications for many of us changed how we share and process information. We were aware of early research that indicated that student learning was deeply affected by the pandemic and that this likely impacted underserved, underresourced and underrepresented students more than their counterparts. Although we thought we may uncover evidence of this through our study from the eyes of practitioners, we were unsure of the extent of this gap and what it meant for schools that were in the middle of the process of building CS education offerings at their schools. Our hope is that this research can provide some empirical groundings for others who are interested in how practitioners perceived the COVID-19 pandemic impacts on students and their schools.
4 Qualitative Survey
This section describes our methodology for creating an instrument to collect qualitative data from open-ended questions deeply rooted in the four CAPE components (Step 1 in Section 3) and provides the results of data collection.
4.1 Methodology
As described for Step 1, we designed the cross-sectional qualitative survey to provide information about the CAPE framework and to capture data to use in the quantitative survey. We planned to send the survey to a broad set of practitioners (e.g., teachers, administrators, counselors) to collect information on how COVID-19 has impacted their CS courses and extracurricular activities, instruction, recruitment and retention efforts.
4.1.1 Instrumentation.
The survey consisted of a high-level description of the CAPE framework and the goals of the study, followed by four parts in total, one for each component of the CAPE framework. Each part contained two open-ended questions–one asking about the past and current impacts of COVID-19 on CS education in context to each component of CAPE (see Figure 4). We used the REDCap survey platform to conduct the survey [18, 19]. The first questions focused on the ways in which the respondent perceived that COVID-19 impacted their school’s or district’s ability to maintain or build capacity (impact on teachers, professional development opportunities, funding/resources, and policies) in computer science education (including computer science-related extracurricular activities) in terms of equity. The second question focused on the ways in which the respondent anticipates that COVID-19 will impact their school’s or district’s ability to maintain or build capacity in computer science education (including computer science-related extracurricular activities) using an equity lens. We customized the survey to randomly choose only one of the components for each participant to answer. In this way, we could maintain coverage across the CAPE components while simultaneously keeping the survey to approximately 10 minutes per response.
Fig. 4.
4.1.2 Participant Recruitment and Characteristics.
We defined participants as high school educators in the U.S. who were involved in CS education. We aimed for 100 participants, 25 for each component. In December 2020 and January 2021, we recruited via social media, through an RPPforCS2 community newsletter, and through two webinars (RPPforCS and ECEP3). Each participant received a $10 gift card if requested.
We closed the survey in January 2021 with 21 responses. Though the submissions were much lower than anticipated, the feedback to the open-ended questions turned out to be rich. Further, although we presented only one CAPE component to each participant, the participants’ responses often applied to more than one component.
The majority of the respondents (95%) selected one of their roles as CS Teacher, while 5% (1) selected principal, 5% selected Coordinator of Makerspace, and 5% preferred not to say. 29% of the respondents stated that they had been in this role for 1 to 5 years, 29% 6–10 years, 19% 11–15 years, 10% 16–20 years, and 14% more than 20 years. Nearly half (48%) stated that they were a woman, 38% man, and 10% preferred not to say. Only 5% stated that they had a disability. Participants responded: White (86%), Asian (10%), Hispanic/Latinx (5%), and preferred not to say (10%).
We asked in which CS networks they engaged. 81% stated that they engaged with CSTA, 14% with ISTE, and 5% with each of ECEP, Research Experiences for Teachers, CSforALL, Digital Promise, Beauty and Joy of Computing, CUE, CBEA, and Code.org. 43% stated that their August-December 2020 semester was offered online, 19% were hybrid (in-person and online), 10% started online and switched to hybrid, and 5% started as in-person and switched to hybrid. 19% also stated Other, specifying “BOTH a full-time (5 days a week) face-to-face school AND a Virtual school online”, “Started as hybrid, and have temporarily switched to online”, “Hybrid but just switched to remote and we’re told we will be back to hybrid in Jan”, and “Combinations - some are fully remote; some are hybrid; some are combos of both; right now it is all remote but will change if the tier color is ok.”
As for the participants’ schools for which participants responded (15 of the 21), 6 were Title I schools4 and 5 (66%) had over 50% of students receiving free or reduced lunch.5 Participants indicated their location as: cities (42%), rural communities (26%), suburbs (21%), and towns (11%). Nine of the schools had over 30% White students, six had over 30% Hispanic students, and one had over 30% Black students.
4.1.3 Data Analysis.
To code each response along the CAPE components, one researcher coded each statement as either the Capacity, Access, Participation, and/or Experience component. Once completed, a second researcher critically critiqued the codes by examining each response and providing detailed feedback. Based on this preliminary analysis, the researchers together settled on three ways to classify the interpretation of feedback: primary impact that relates directly to the feedback provided, a secondary inference from the primary impact, or a comment that may help establish the qualitative interview protocol for the last leg of this overall study. As an example, the feedback statement “Covid has made it difficult to offer extra help for students who don’t normally reach out for help, but in a classroom you can go to them and help them without drawing attention to them.” had one primary impact that we aligned with the Capacity component as “Capacity to provide extra help to students”. We considered the inferences from this and were able to provide two secondary impacts under Experience, “Student assignment completion rates” and “Student grades”, and one statement under Experience that we will consider when developing the interview protocols to digger deeper into why this occurs, “Students less likely to engage in help-seeking behaviors”. Again, one researcher classified the participant responses according to these codes and the second reviewer critically critiqued them and provided detailed feedback. The researchers worked together to address all concerns.
4.1.4 Evidence of Reliability and Validity.
For this instrument, we relied on a combination of external and internal face validity to ensure that we posed meaningful questions that align with the CAPE framework. First, two members of the team created the questionnaire. Then, we solicited feedback from those involved with the creation of the CAPE framework provided detailed feedback on the questions. Once we reviewed these statements and made changes, four members of our internal team then reviewed multiple revisions of the instrument–that is, our team provided feedback, made modifications, then the instrument was sent to the team for further review. We repeated this process three times until all of our collective concerns were addressed.
4.2 Results
In total, we coded 35 non-unique primary impacts and secondary inferences for Capacity, 7 for 379 Access, 23 for Participation, and 50 for experience (Table 1). Since the majority (95%) of the responses came from teachers, it makes sense that the impacts are more heavily-oriented towards student experiences. However, we coded only 10 primary impacts for experience and 40 were secondary inferences (e.g., “Students grades”). Interestingly, the majority of primary impacts were rooted in Capacity across all of the participants’ feedback, despite the component for which the survey asked them to provide feedback. Overall, the number of primary impacts with respect to future implications (35) was much lower than the past and current implications (48).
Table 1.
| Participant | Researcher Response Classification | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Feedback | Capacity | Access | Participation | Experience | ||||||||
| Category | P | S | IP | P | S | IP | P | S | IP | P | S | IP |
| Past & Current | ||||||||||||
| Capacity | 6 | 2 | 0 | 1 | 0 | 0 | 3 | 2 | 0 | 3 | 2 | 0 |
| Access | 5 | 4 | 0 | 2 | 0 | 0 | 2 | 4 | 0 | 0 | 14 | 1 |
| Participation | 4 | 3 | 0 | 2 | 1 | 0 | 2 | 6 | 0 | 3 | 5 | 0 |
| Experience | 8 | 3 | 0 | 1 | 0 | 0 | 2 | 2 | 0 | 4 | 19 | 0 |
| Subtotal | 23 | 12 | 0 | 6 | 1 | 0 | 9 | 14 | 0 | 10 | 40 | 1 |
| Future | ||||||||||||
| Capacity | 1 | 0 | 0 | 1 | 0 | 0 | 2 | 4 | 0 | 0 | 1 | 0 |
| Access | 4 | 2 | 1 | 1 | 2 | 0 | 0 | 5 | 0 | 0 | 7 | 0 |
| Participation | 7 | 0 | 0 | 2 | 0 | 0 | 3 | 5 | 0 | 4 | 2 | 0 |
| Experience | 6 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 4 | 11 | 0 |
| Subtotal | 18 | 2 | 1 | 4 | 3 | 0 | 5 | 15 | 0 | 8 | 21 | 0 |
Table 1. Response Classification
The participant feedback category reflects the CAPE component with which the participant was prompted to reflect. The researcher response classification reflects that even though the participant was given only one component, their responses (P = Primary) sometimes belonged in other components. Secondary inferences (S) were coded by the researchers. Comments that reflected an underlying reason for the impacts were tagged (IP) to be considered for Interview Protocol for a planned qualitative study.
Table 2.
| Capacity - Changes in: | |
| Capacity for qualified teachers to teach CS | |
| Capacity for schools to offer CS extracurricular activities | |
| Capacity for schools to offer non-CS extracurricular activities | |
| Capacity to ensure that teachers are fully supported to teach CS | |
| Capacity to offer CS education | |
| Capacity to offer devices that meet hardware/software requirements for CS instruction to students needing them | |
| Capacity to offer digital tools used in virtual instruction | |
| Capacity to offer instructional help to students | |
| Capacity to offer reliable internet with appropriate bandwidth to students | |
| Capacity to offer software with teacher management tools to teachers | |
| Capacity to offer stable environments for learning | |
| Capacity to offer support to parents of students | |
| Capacity to offer technology infrastructure for students (e.g., devices to students, WiFi access to students) | |
| Capacity to provide extra help to students | |
| Funding for CS education | |
| Resources for CS education | |
| Effectiveness of online teacher PD | |
| For those offering asynchronous, online courses, capacity to offer quality of instruction comparable to in-person | |
| For those offering asynchronous, online courses, capacity to offer teacher assistance comparable to in-person | |
| Online teacher PD access | |
| Quality of in-person instruction compared to past in-person instruction | |
| Quality of virtual instruction compared to past in-person instruction | |
| Support for Teacher PD | |
| Access - Changes in: | |
| Number of CS courses offered | |
| Number of CS extracurricular activities offered | |
| Number of non-CS extracurricular activities offered | |
We removed duplicate responses, which reduced the number of items for past and current impacts to 46 (Capacity 23, Access 3, Participation 9, and Experience 11) and future impacts to 52 (Capacity 15, Access 3, Participation 8, and Experience 21) (see Tables 2 and 3. Given the rich dataset that these limited number of participants provided and the adequate representation of participants and their schools (though we would like to have seen more schools with Black students represented), we were assured that we would have sufficient data to use to build our quantitative survey.
Table 3.
| Participation - Changes in: | |
| Number of students dropping a CS course | |
| In schools with in-person and virtual classes, underrepresented students were more likely to enroll in virtual courses | |
| Number of students enrolled in in-person instruction compared to previous years | |
| Number of students enrolled in CS courses | |
| Number of students participating in CS related extracurricular activities | |
| Number of students participating in CS honor society | |
| Number of students participating in non-CS extracurricular | |
| Number of students participating in multiple club meetings | |
| Experience - Changes in: | |
| Content knowledge students gained in CS classes | |
| Grades students received in CS classes | |
| Student attendance in CS classes | |
| Students’ completion of homework assignments | |
| Student interest in CS | |
| Engagement in help-seeking behaviors | |
| Student willingness to share their knowledge | |
| Number of hours students received CS instruction | |
| Number of students taking Advanced Placement (AP) CS courses | |
| Student engagement in CS Courses | |
| Student engagement in extra-curricular activities | |
| Student engagement with pair programming | |
5 Quantitative Survey
In this section, we describe our methodology for the quantitative study and provide the results.6
5.1 Methodology
After we extracted the impacts from the first survey, we examined the responses for overlap, completeness, and novelty. We also considered the list of impacts from our previous research on using CAPE to disaggregate a pilot project to check for omissions from participants, adding 12 items to the list. We then went through each item to produce consistently phrased items that made sense for the category for which it appeared and grouped similar items (Table 4).
Table 4.
| Component | Categories | # of Items | Cronbach’s Alpha (5-scales) | Cronbach’s Alpha (3-scales) |
|---|---|---|---|---|
| Capacity | Funding, Policy, & Curriculum | 9 | 0.88 | 0.89 |
| Physical Resources | 5 | 0.79 | 0.86 | |
| Human Resources | 10 | 0.74 | 0.79 | |
| Access | (Access) | 47 | 0.26 | 0.58 |
| Participation | (Participation) | 10 | 0.57 | 0.83 |
| Experience | Learning | 6 | 0.37 | 0.65 |
| Engagement | 7 | 0.77 | 0.91 | |
| Other noncognitive factors | 5 | 0.62 | 0.76 | |
| CS AP Exams | 2 | n/a | n/a |
Table 4. CAPE Components Segregated by Categories with Reliability Measures
Three scales indicate only Increased, Stayed the Same, and Decreased scales were used (Unsure and Not applicable were removed) and rows with blank data were removed.
Our goal was to keep the survey to a 10-minute response time. We condensed items that were somewhat similar or seemed too narrow to reduce the number of items on the survey. We also eliminated the “future impacts” from this survey and added questions about anticipated future impacts to the qualitative protocol that will take place in the future. The impact of COVID-19 on the future CS educational ecosystem would be unquestionably speculative given the timing of those in the U.S. getting vaccines and the potential impact from COVID-19 variants.
Based on the open-ended feedback, we chose to have the scale responses be Increased, Stayed the same, Decreased, Unsure, and Not Applicable. To limit the length of the survey, we set up the survey to select only two components to administer to the participants. Since the participants with roles other than a teacher (e.g., administrator, counselor, etc.) would be most familiar with items related to Capacity and Access, any participant who selected those roles received items from those two components. Since we anticipated that there would be more participants who held the role of a teacher than other roles, those who selected teacher as their primary role would have a 50-50 chance of either receiving the Capacity and Access items or the Participation and Experience items. In this way, we anticipated having thorough coverage, and the survey itself would remain somewhat brief (10-15 minutes to complete). Once the survey was complete, participants could link to a second survey to enter a drawing for one of two $75 gift cards. To prevent bot responses, we used reCaptcha and set up a randomly-generated, simple addition question.
5.1.1 Instrumentation.
Similar to the first survey, we defined participants as high school educators in the United States who were involved in CS education, either as a teacher, administrator, counselor, curriculum designer, or similar role. In 2019, there were 1,050,800 public and private high school teacher positions [44]. Although there are several subject areas that high schools can offer, we examined this with the assumption that 1% of teachers, or 10,508, taught CS. It is also estimated that 47% of U.S. high schools offer CS in some form [6], and there are 26,727 high schools as of 2018 [45], leaving 12,561 schools offering CS. Assuming one CS teacher per high school and averaging the difference of the two, we get 11,534. Using this as a basis and an online sample size calculator, to reach a 95% confidence level with a 5% margin of error, we needed a sample size of 384 for each component or a total of 742 participants.
Our recruitment strategy included the following: recruiting in organizations involved with CS teachers (e.g., CSTA, the RPPforCS community, the ECEP community, the ACM SIGCSE listserv), contacting researchers we knew who were involved in studies involving CS education, and contacting other organizations involved in CS education initiatives.
This strategy does not attempt to query schools that do not have existing CS education in place, nor do we feel as confident about representation from schools where CS educators are not connected with CS networks (like CSTA). We attempt to ascertain the networks of the CS educators in one of the demographic questions.
5.1.2 Data Collection and Cleaning.
We opened the survey in February 2021 and held it open for nearly three weeks, receiving 252 responses. We removed 66 incomplete surveys. We removed one response in which the simple math question was incorrect. This brought our final number of responses to 185. Of these, we coded 98 Capacity and Access from responses and 87 Participation and Experience. This number falls short of the sample size needed for a 5% confidence interval we hoped for. However, it does achieve a 10% confidence interval.
5.1.3 Data Analysis.
To analyze the data, we used PSPP [51], Excel and CalculatorSoup for conducting descriptive statistics and examined differences between schools serving students from low-income and higher income families. A participants’ answers in all questions fell into the “low-income” if a participant’s response indicate that over 50% of students received free or reduced lunch.
We calculated the percentage of responses against the 3-response scale (Increased = 1, Stayed the Same = 2, and Decreased = 3) for schools serving both low-income and non-low (higher) income. This gave us a perspective of the impact of COVID-19 on CS Education across the country. To provide context to the numbers, we ran a one sample \(t-test\) against the value “2”, which represented the Stayed the Same response. We used p-values to indicate how significant the Increase or Decrease was from the Stayed the Same response, while also reporting confidence intervals and resulting \(t\) values. In other words, we are examining whether the growth or decline is significantly different from no change.
For comparison across the two types of schools, we placed the descriptive data in a stacked bar chart. We also conducted a chi-square analysis using only the Increased (1), Stayed the Same (2), and Decreased (3) values in the test for significance. When conducting the chi-square analysis, we measured the difference between Stayed the Same (2) and Increase (1) across the two types of schools and we also measured the difference between Stayed the Same (2) and Decreased (3), treating these as categorical data.
5.1.4 Evidence of Reliability and Validity.
For this instrument, one of the researchers conducted internal face validity among the four members of the team. The researcher modified the survey items based on the feedback and sent for a second review and feedback cycle. For reliability, we conducted a Cronbach’s alpha test for the groupings presented in Table 4 on all five scales and with only the three scales of Increased, Stayed the Same, and Decreased. The majority of items are close to or above .70, indicating strong reliability between the items in these groups. In our results, we present items individually.
5.2 Results
We present a description of the participant demography followed by the results from each CAPE component. For the comparative analyses in this section, we removed responses of Unsure and Not Applicable.
5.2.1 Demographic Data.
The majority of the participants were teachers (\(n^{total}\) = 161 (88%); \(n^{C\&A}\) = 75 (77%); \(n^{P\&E}\) = 86 (100%)). With respect to the time spent in the current positions, the majority of participants have been in their positions between 1–5 years (\(n^{total}\) = 73 (42%); \(n^{C\&A}\) = 41 (43%); \(n^{P\&E}\) = 32 (40%)). With respect to gender, the majority identified as women (\(n^{total}\) = 99 (57%); \(n^{C\&A}\) = 57 (60%); \(n^{P\&E}\) = 42 (53%)), though not overwhelmingly. With respect to race/ethnicity, the majority identified as White (\(n^{total}\) = 156 (84%); \(n^{C\&A}\) = 81 (80%); \(n^{P\&E}\) = 75 (88%)). In summary, most participants identified as women - though not overwhelmingly - and most identified as White.
We also asked about the CS networks with which participants affiliated to gauge their involvement with organizations that support K-12 CS education. Only a few respondents were not affiliated with a CS network (\(n^{total}\) = 31 (14%); \(n^{C\&A}\) = 23 (19%); \(n^{P\&E}\) = 8 (8%)). The majority of participants were affiliated with the CS Teachers Association (CSTA), (\(n^{total}\) = 123 (54%); \(n^{C\&A}\) = 58 (48%); \(n^{P\&E}\) = 65 (62%)).
The composition of the schools was balanced between boys and girls, though there were several respondents from all-girls or all-boys schools. With respect to rural geographic locations, approximately one-third of respondents indicated that 60% or greater of their students were from rural locations (\(n^{total}\) = 61 (33%); \(n^{C\&A}\) = 29 (30%); \(n^{P\&E}\) = 32 (37%)). With respect to race/ethnicity, the majority of respondents indicated that their school was composed of 81% or higher White students (\(n^{total}\) = 97 (55%); \(n^{C\&A}\) = 54 (57%); \(n^{P\&E}\) = 42 (53%)). We asked participants how their school was offering their current courses/classes. The majority of respondents reported that they offered a hybrid model (in-person and online) (\(n^{total}\) = 96 (52%); \(n^{C\&A}\) = 32 (34%); \(n^{P\&E}\) = 26 (33%)).
To identify if a participant’s school served students from lower-income families, we asked, “Does your school historically have students that meet greater than 50% free or reduced lunch guidelines?”. The responses were nearly evenly split: Yes (\(n^{total}\) = 89 (51%); \(n^{C\&A}\) = 48 (51%); \(n^{P\&E}\) = 41 (51%)). For those that answered Yes, we categorized their remaining responses to reflect schools serving students from low-income families. If they selected No, we categorized their remaining responses to reflect “higher income” families.
5.2.2 Capacity.
We grouped Capacity into three categories: Funding, Policy & Curriculum, Physical Resources, and Human Resources. All three sections are highlighted here.
Funding, Policy & Curriculum. For the Funding, Policy & Curriculum category, we first analyzed the combined data from participants from both types of schools (Figure 5).7 For statistically significant increases and decreases when examining the combined data from all participants (Table 5), we found five items with significant increases: two with extremely statistically significant increases (Strategies to make CS curriculum more equitable and Strategies to improve CS curriculum), one with a very statistically significant increase (Plans to add additional CS courses), and two with statistically significant increases (Strategies to recruit more diverse students into CS and Strategies to add CS A or CS Principles courses).
Fig. 5.
Table 5.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Funding for CS Education | 9% | 77% | 14% | t(80) = 0.69 | –0.07, 0.14 | 0.495 |
| State, district, or school initiatives related to CS education | 25% | 54% | 20% | t(84) = –0.80 | –0.21, 0.09 | 0.427 |
| CS graduation requirements | 9% | 85% | 6% | H(1) = 0.26 | 0.608 | |
| Plans to add additional CS courses | 32% | 51% | 15% | t(90) = –2.61 | –0.33, –0.04 | 0.010 |
| Strategies to make CS curriculum more equitable | 34% | 58% | 9% | t(85) = –3.55 | –0.36, –0.10 | 0.001 |
| Strategies to improve CS curriculum | 40% | 48% | 11% | t(90) = –4.30 | –0.43, –0.16 | 0.001 |
| Strategies to recruit more diverse students into CS | 30% | 56% | 14% | t(87) = –2.47 | –0.31, –0.03 | 0.015 |
| Strategies to integrate CS into other disciplines | 15% | 60% | 21% | t(84) = 0.34 | –0.11, 0.16 | 0.734 |
| Strategies to add CS A or CS Principles courses | 28% | 59% | 13% | t(79) = –2.32 | –0.30, –0.02 | 0.023 |
Table 5. Capacity Measured by Participants’ Perceptions of Funding, Policy & Curriculum
Values reflect combined responses of participants for schools serving students from low- and higher-income families.
Next, we conducted a chi-square analysis to determine whether outcomes differed based on school type (low- vs. high-income). The results revealed that there were no statistically significant differences based on school type.
Physical Resources. Combining the data from participants from both types of schools, we found that physical resources decreased extremely statistically significantly for Stable learning environments, very significantly for Physical tools used to teach CS, and significantly for Reliable internet for learning CS (see Figure 6 and Table 6). The results of the chi-square revealed that there were no statistically significant differences based on school type.
Fig. 6.
Table 6.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Stable environments for learning | 11% | 33% | 56% | t(97) = 6.44 | 0.31, 0.59 | 0.001 |
| Reliable internet with appropriate bandwidth suitable for learning CS for students who need it | 27% | 25% | 48% | t(95) = 2.56 | 0.05, 0.39 | 0.012 |
| Devices that meet hardware and software requirements for CS instruction to students who need them | 23% | 35% | 38% | t(95) = 1.56 | 0.03, 0.29 | 0.122 |
| Physical tools used to teach CS | 12% | 54% | 32% | t(90) = 2.75 | 0.05, 0.32 | 0.007 |
| Digital tools used to teach CS | 23% | 54% | 22% | t(95) = –0.45 | –0.17, 0.11 | 0.657 |
Table 6. Capacity Measured by Participants’ Perceptions of Physical Resources
Values reflect combined responses of participants at schools serving students from low- and higher-income families.
Human Resources. With respect to human resources, we found five (and one approaching) statistically significant changes when we combined the data from participants from both types of schools (Figure 7, Table 7). Three items (Faculty/staff availability to offer CS-related extracurricular activities, Faculty/staff availability to discuss taking CS courses with parent/guardian, and Faculty/staff availability to train parents of CS students), all saw extremely statistically significant decreases.
Fig. 7.
Table 7.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Teachers qualified to teach CS | 11% | 82% | 7% | H(1) = 0.74 | 0.388 | |
| Teacher ability to offer high-quality CS instruction | 13% | 64% | 23% | t(88) = 1.06 | –0.06, 0.19 | 0.291 |
| Teacher availability to offer extra instructional help to students | 15% | 44% | 41% | t(90) = 3.20 | 0.09, 0.39 | 0.002 |
| Faculty/staff availability to offer CS-related ec activities | 8% | 42% | 51% | t(87) = 6.01 | 0.28, 0.56 | 0.001 |
| Faculty/staff availability to encourage CS participation | 19% | 45% | 35% | t(87) = 1.75 | –0.02, 0.29 | 0.083 |
| Number of students who received information about CS courses/CTE pathways | 32% | 42% | 27% | t(67) = –0.82 | –0.25, 0.11 | 0.415 |
| Faculty/staff availability to attend CS PD | 20% | 47% | 33% | t(91) = 2.27 | 0.02, 0.33 | 0.026 |
| Faculty/staff availability to discuss taking CS courses w/ guardians | 4% | 51% | 45% | t(84) = 6.29 | 0.28, 0.54 | 0.001 |
| Faculty/staff availability to train parents of CS student | 3% | 45% | 52% | t(70) = 6.28 | 0.31, 0.59 | 0.001 |
| Specialized training to teachers on equity | 17% | 51% | 32% | t(86) = 1.97 | 0.00, 0.30 | 0.052 |
Table 7. Capacity Measured by Participants’ Perceptions of Human Resources
Values reflect combined responses of participants at schools serving students from low- and higher-income families.
One item, Teacher availability to offer extra instructional help to students, had a very statistically significant decrease, and one item, Faculty/staff availability to attend CS professional development had a statistically significant decrease. A sixth item, Faculty/staff availability to encourage students to take CS courses, was approaching a statistically significant decrease.
The results of the chi-square revealed that the decrease in the Number of students who received information about CS courses/CTE pathways was significantly higher among schools serving students from low-income families \((\chi ^2(2, {\it N} = 60) = 8.61, {\it p} = 0.01)\).
5.2.3 Access.
For Access (Figure 8 and Table 8), when combining data from participants from both types of schools, we found statistically significant differences across every item, with a mix of increases and decreases. Two items had extremely significant decreases (Number of CS-related extracurricular activities offered and Number of non-CS related extracurricular activities offered. One item had an extremely significant increase (Number of classes conflicting with CS classes) one had a very significant increase (Number of CS courses offered).
Fig. 8.
Table 8.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Number of CS courses offered | 27% | 69% | 4% | t(90) = 3.01 | –0.29, –0.06 | <0.01 |
| Number of CS related extracurricular activities offered | 11% | 80% | 9% | t(86) = 6.91 | 0.32, 0.58 | <0.01 |
| Number of non-CS related extracurricular activities offered | 5% | 73% | 22% | t(84) = 10.02 | 0.47, 0.70 | <0.01 |
| Fees to take CS courses over the last 12 months | 15% | 69% | 15% | H(1) = 1.04 | 0.31 | |
| Number of classes conflicting w/ CS classes | 32% | 50% | 18% | t(58) = 5.49 | –0.51, –0.24 | <0.01 |
Table 8. Access Measured by Participants’ Perceptions
Values reflect combined responses of participants at schools serving students from low- and higher-income families.
With respect to enrollment fees, which can be a barrier to access for some students, we asked if the participant’s school required students to pay any fees to take CS courses in either 2019 or 2020. Nine schools serving students from higher income families and four from lower-income families reported charging additional fees, and there was no statistically significant difference between low income and higher income schools. For these 13, we found that one school (18%) saw an increase in fees, while all others remained the same.
The results of the chi-square revealed that there were no statistically significant differences based on school type.
5.2.4 Participation.
With respect to Enrollment (Figure 9 and Table 9) for the participant data from both schools combined, we found no statistically significant changes in any of items. We found one item that approached a statistical significance decrease, Number of students enrolled in CS A courses.
Fig. 9.
Table 9.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Number of students enrolled in CS courses | 32% | 44% | 24% | t(81) = –0.88 | –0.24, 0.09 | 0.38 |
| Number of students enrolled in CS A courses | 18% | 44% | 38% | t(49) = 1.94 | –0.01, 0.41 | 0.06 |
| Number of students enrolled in CS Principles courses | 29% | 42% | 29% | t(51) = 0.00 | –0.20, 0.20 | 1.00 |
| Number of girls enrolled in CS classes | 24% | 51% | 26% | t(84) = 0.26 | –0.13, 0.18 | 0.76 |
| Number of Black, Hispanic, Indigenous students enrolled in CS classes | 19% | 67% | 15% | t(80) = 0.62 | –0.17, 0.09 | <0.001 |
| Number of students participating in CS related extracurricular activities | 10% | 32% | 58% | t(69) = 6.56 | 0.36, 0.67 | <0.01 |
| These two items are not specific to CS, but are used for comparisons. | ||||||
| Number of students participating in non-CS related extracurricular activities | 4% | 25% | 72% | t(64) = 10.23 | 0.54, 0.81 | <0.01 |
| Number of students participating in multiple extracurricular activities | 5% | 19% | 76% | t(70) = 10.27 | 0.54, 0.81 | <0.01 |
Table 9. Student Participation Measured by Participants’ Perceptions
Values reflect combined responses of participants at schools serving students from low- and higher-income families.
We found very statistically significant decreases across three items (Number of students participating in CS-related extracurricular activities, Non-CS related extracurricular activities, and Multiple extracurricular activities).
The results of the chi-square revealed that decreases in the Number of girls enrolled in CS classes were significantly higher among schools serving students from low-income families \((\chi ^2(2, {\it N} = 74) = 6.48, {\it p} = 0.04)\).
5.2.5 Experience.
We grouped Experience into four categories: Learning, Engagement, Other non-cognitive factors, and Taking AP exams.
Experiences in Learning. In Learning (Figure 10 and Table 10), we found several statistically significant decreases among four items. Two items, Completion of CS homework assignments and Number of institutional hours in CS students received, indicated extremely statistically significant decreases. One item, Content knowledge students gained in CS classes, indicated a very statistically significant decrease and one item Grades given in CS classes indicated a statistically significant decrease.
Fig. 10.
Table 10.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Content knowledge students gained in CS classes | 21% | 32% | 48% | t(80) = 3.28 | 0.11, 0.46 | <0.01 |
| Grades given in CS classes | 17% | 50% | 33% | t(79) = 2.27 | 0.02, 0.33 | 0.03 |
| Completion of CS homework assignments | 10% | 24% | 66% | t(78) = 8.13 | 0.45, 0.74 | <0.01 |
| Number of instructional hours in CS students received | 7% | 31% | 62% | t(81) = 8.12 | 0.41, 0.68 | <0.01 |
| Number of students receiving college credit for dual-credit CS courses | 15% | 48% | 36% | t(367) = 1.36 | –0.08, 0.39 | 0.18 |
| Number of students achieving awards in CS | 18% | 49% | 33% | t(52) = 1.23 | –0.07, 0.30 | 0.22 |
Table 10. Experience Measured by Participants’ Perceptions of Student Learning
Values reflect combined responses of participants for schools serving students from low- and higher-income families.
Comparisons between schools serving students from low- vs. higher-income families revealed that decreases in the Number of students receiving College Credit for Dual-Credit CS Courses were significantly higher among schools serving students from low-income families \((\chi ^2(2), {\it N} = 33) = 13.69, {\it p} \lt .01)\).
Engagement by Experiences. For Engagement (Figure 11 and Table 11), we found extremely statistically significant decreases in engagement activities across the board when combining the data from participants from both types of schools. When comparing differences between school types, the results of the chi-square revealed statistically significant differences in Attendance in CS Classes. Schools serving students from low-income families showed significant higher decreases in attendance than did those serving high-income families \((\chi ^2(2, {\it N} = 72) = 9.32, {\it p} \lt .01)\).
Fig. 11.
Table 11.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Willingness to share their knowledge during class | 15% | 23% | 62% | t(79) = 5.12 | 0.26, 0.59 | <0.01 |
| Engagement during CS classes | 18% | 15% | 68% | t(79) = 5.72 | 0.32, 0.66 | <0.01 |
| Engagement with other students | 12% | 14% | 74% | t(78) = 7.48 | 0.44, 0.75 | <0.01 |
| Engagement in help-seeking behaviors | 22% | 16% | 62% | t(78) = 4.24 | 0.21, 0.58 | <0.01 |
| Engagement during pair programming exercises | 14% | 22% | 64% | t(69) = 5.51 | 0.31, 0.66 | <0.01 |
| Engagement during CS related extracurricular activities | 9% | 26% | 65% | t(59) = 6.56 | 0.38, 0.72 | <0.01 |
| Attendance in CS classes | % | % | % | t(82) = 4.20 | 0.16, 0.46 | <0.01 |
Table 11. Experience Measured by Participants’ Perceptions of Student Engagement
Values reflect combined responses of participants for schools serving students from low- and higher-income families.
Engagement Measured by Noncognitive Factors. For the remaining noncognitive factors (Figure 12 and Table 12), when analyzing the data from both types of schools, we found extremely statistically significant increases across two items: Understanding of the relevance of technology (39%) and Confidence using technology.
Fig. 12.
Table 12.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Interest in CS | 25% | 52% | 23% | t(69) = –0.35 | –0.19, 0.13 | 0.73 |
| Belonging in CS courses | 23% | 55% | 23% | t(67) = –0.18 | –0.17, 0.15 | 0.85 |
| Understanding the relevance of technology | 39% | 51% | 10% | t(75) = –4.02 | –0.43, –0.15 | <0.01 |
| Confidence using technology | 41% | 46% | 13% | t(75) = –3.47 | –0.41, –0.11 | <0.01 |
| Number of students interested in taking additional CS courses | 29% | 53% | 18% | t(61) = –1.53 | –0.30, 0.04 | 0.13 |
Table 12. Experience Measured by Participants’ Perceptions of Student Interest, Belongingness, and Other Factors
Values reflect combined responses of participants for schools serving students from low- and higher-income families.
Additionally, the results of the chi-square revealed that schools serving students from lower-income families had significantly larger decreases in the Number of students interested in taking additional CS courses, while higher-income serving schools had significantly higher increases in this area \((\chi ^2(2, {\it N} = 60) = 11.61, {\it p} \lt .01)\).
Advanced Placement (AP) CS exams. For students taking AP CS exams (Figure 13), again, we found that the item Number of students taking CS AP Exams saw a very statistically significant decrease. We also found that the item Number of students taking CS Principles Exams was approaching statistically significant decrease (Table 13). The results of the chi-square revealed that there were no statistically significant differences based on school type.
Fig. 13.
Table 13.
| Item | Increased | Same | Decreased | \(t-test\) | \(CI\) | \(p\) |
|---|---|---|---|---|---|---|
| Number of students taking AP CS A exam | 15% | 36% | 49% | t(41) = 3.19 | 0.13, 0.58 | <0.01 |
| Number of students taking AP CS Principles exam | 16% | 42% | 42% | t(50) = 1.94 | 0.00, 0.40 | 0.06 |
6 Discussion
The results presented in the previous section provide empirical evidence of teachers’ perceptions of the pandemic’s impact on student learning, but also the impact on the capacity to offer computer science education to all students. Overall, the student impact findings match what has been stated in early reports–that student learning has indeed been heavily impacted. However, we have learned that students’ understanding of the value and relevance of CS has increased. In this section, we take a deeper, reflective look at the findings.
6.1 Observations
6.1.1 Capacity.
Some districts and schools often provided resources like Wifi and hardware devices to students, with one teacher stating that the “COVID pandemic has had no significant impact on my school’s ability to maintain capacity for CS education. There has been no effect on funding, resources, or access to professional development.” We were surprised to learn that the quantitative data supported this, and that there were no significant decreases related to Funding, Policy & Curriculum. Remarkably, we found five of the nine items with statistically significant growth, two of which were related to equity (Strategies to make CS curriculum more equitable and recruit more diverse students into CS), while the other four saw no change. One reason for this increase may be related to any urgency administrators may have felt after a year in which racial tensions and unrest were growing within the U.S. after several highly-publicized police killings. Another reason may be that administrators felt an urgency after experiencing the necessity and relevance of computers and technology to teach students–a statement of which was made in responses to the first survey of our study. More states are also requiring schools teach computer science [6], and this could have also been reflected in this continued growth.
Schools’ capacity to offer physical resources needed for teaching and learning in a CS course significantly decreased in three of the five categories. These indicated the lack of ability for schools to offer stable learning environments for CS students prior to the pandemic, notably as it relates to physical tools (e.g., robots) for teaching CS and reliable internet. These results are in line with early research [23] as well as multiple reports from the U.S. media that stressed some of the real-time impacts of remote learning on students [46].
Examining capacity with respect to human resources, we found that schools experienced statistically significant decreases across five items, with a sixth approaching statistical significance. Given this, it is safe to state that human resources were significantly impacted by the COVID-19 pandemic, which supports earlier findings [8].
Some items, like Capacity to offer extra instructional help to students and Faculty/staff availability to attend CS professional development, are more nuanced issues that have not yet appeared in research, but nevertheless are essential for academic achievement [25] and for recruiting students into elective CS courses.
When comparing outcomes based on school type, we found that practitioners perspectives’ indicated that schools serving students from lower-income families showed statistically significant decreases in the Number of students who received information about CS courses/CTE pathways compared to their higher-income counterparts.
6.1.2 Access.
Variability in Access was noted in the teacher feedback, with one teacher stating that “Majority of our extracurricular clubs have stopped.” while another stating that “On a positive note, due to the relative ease of organizing virtual meetings and the lack of competing sports and social activities, we’ve had far more extracurricular engagement in our after-school CS clubs, including from under-represented populations.” We found that the quantitative data also showed a mix of increases and decreases. There was a significant increase in the number of CS courses offered, perhaps reflecting on the trend for high schools to add more CS courses to their curriculum [6]. This aligns with the increases in Capacity findings for Funding, Policy, & Curriculum on which Access is entirely dependent and reflects on many schools’ efforts to continue incorporating CS into their curriculum.
We found that the number of classes conflicting with CS classes increased significantly. Decisions made at the Capacity level may have included shifts in schedules to accommodate on-line learning needs, and this may have prohibited students from taking CS courses during the fall and spring semesters [23]. We also found significant decreases in the number of CS related and non-CS related extracurricular activities offered, which aligns with previous findings [4, 47].
We note that with the Capacity measures showing an increase across funding, policy and curriculum, the Access measures indicate that these increases were not enough to ensure that CS education offerings were maintained across the board. This may indicate that the intent of growing CS education was faced with significant challenges in providing the human and physical resources needed to deliver pre-pandemic levels of instruction.
When comparing the two types of schools, we found no statistically significant differences between low- and higher-income serving schools, indicating Access was impacted similarly in both.
6.1.3 Participation.
Among some schools, we found only one item had a near statistically significant decrease in the number of students enrolled in CS A courses, which is similar to results found in one earlier study [27]. When considering some of the impact on Access to informal education (Section 6.1.2), we see that student Participation in informal CS education significantly decreased. These findings confirm the dependency of Participation on Access within the CAPE framework.
During the qualitative portion of this study, one of the teachers remarked that “Most of my diverse students (Asian, Black, Latino, and female) are in the virtual school with ChromeBooks, where my counterpart in the virtual school struggles to provide the same quality curricula on ChromeBooks as I do with Desktops.” This aligns with the findings from the IES ongoing data collection in which Black and Hispanic students were enrolled in fully remote learning models at higher rates than average, coupled with the fact that those who are learning remotely face higher obstacles (e.g., stable learning environments, and in the case of CS education, adequate technology to support learning) [21]. For students from low-income families, “low income students were more likely to have to take care of younger siblings during the school day” while facing other obstacles like less likely to have parental support.
Participation outcomes based on school type differed in one area. Practitioners’ perspectives indicated that low-income serving schools showed greater decreases in the Number of girls enrolled in CS classes than higher-income serving schools.
6.1.4 Experience.
Engagement was the hardest hit across all of the categories that we measured, with statistically significant decreases across all items. We found these decreases among both low-income and higher-income serving schools. This, no doubt, has impacted and influenced students’ experiences learning CS. We found very significant decreases in learning, including the number of instructional hours CS students received and their completion of CS homework assignments. We also found significant decreases in grades given in CS classes, content knowledge students gained in CS classes, and the number of students receiving college credit for dual-credit CS courses. These changes were comparable across low-income and higher income schools. The reduction in learning and impact on knowledge gains has now been seen across numerous other studies [14, 23, 37, 38] and will certainly impact CS content knowledge gains for these students.
Additionally, when analyzing differences in outcomes between low- and higher-income serving schools, we found a statistically significant difference in the Number of students receiving college credit for dual-credit CS courses and in Attendance in CS classes. Low-income serving schools saw significant decreases in both areas, which unfortunately furthers the digital divide [6, 24, 46, 48].
For AP CS exams, again, we found a significant decrease in the number of students taking these exams. This is counter to the growth College Board saw for AP exams in 2020, where they found a 1% growth in students taking CS A and a 21% growth in CSP exams [7]. This requires further investigation. Also noteworthy is the applicability of both AP CS measures, including whether low-income serving schools are providing support for students to take AP CS exams. Given the reduction in learning experiences in CS, it will be interesting to see how 2021 AP exams are impacted–both in terms of students taking the exam and also scores.
For the noncognitive factors, we found statistically significant increases in the interest in CS, understanding of the relevance of technology, confidence using technology, and number of students interested in taking additional CS courses. This reflected our qualitative feedback from the first survey in which responses indicated that students became steeped in the technology by attending online classes and this piqued their interest in technology by bringing home its relevance. This is interesting, particularly in light of (1) Mooney and Becker’s work investigating belongingness in CS, particularly with the nuances that surrounded some of their findings, and (2) our findings that indicate that learning experiences and engagement were so adversely impacted [31].
Additionally, the impact on student experiences were differed in three categories based on the type of school they attended. For schools serving students from low-income families:
•
There were greater decreases in the number of students who received college credit for dual-credit CS courses.
•
There were greater decreases in the attendance in schools serving students from low-income families.
•
There were greater decreases in the number of students interested in taking additional CS courses in schools serving students from low-income families.
•
While schools serving students from higher-income families saw larger increases in the Number of Students Interested in Taking Additional CS Courses, schools serving students from lower income families saw significantly larger decreases.
These results capture some of the variabilities across the different schools. As one teacher put it, “We have actually thrived with our CS program in COVID. Three years ago, 11 kids in the school of 850 took AP CS. This year, when we are all remote, over 25 students are taking CS with around 200 taking AP CS. The online resources have made it thrive in a remote setting.” Other feedback continued to stress the difficulties that students from low income families faced, like caring for siblings or poorer Internet access. As one teacher stated when predicting the future impacts of COVID-19 on CS education, “Really can’t predict – my guess is that it will be a wash overall, but the individual experiences will be all over the map.”
6.2 Interpreting 2020 CS Education Research Studies
As mentioned earlier, there is a U.S. heightened perceptional awareness of Black students’ experiences which may have changed the focus more onto equity over this past year. It is still too early to tell if it will persist or be short-lived. For those conducting studies, we recommend taking this into account–that increases or decreases in students’ experiences can be impacted by efforts of educators to address inequities. This dual-impact on education makes it harder to tease apart the COVID-19 impacts in the U.S., and we recommend researchers pay attention to both phenomenon when reflecting on their own studies.
Based on our results and early findings from others like Kuhfeld et al., there will be a delayed impact, potentially scaffolded, with additional future impacts that we need to consider [23]. We recommend that researchers take account that the pandemic’s impact on students’ learning experiences are significant. We can also anticipate that some students will be in a more challenged position than they would be had the pandemic not occurred.
Our study also shows that there is a significant reduction in informal learning, and the impacts of this may not be measurable for some time. After school and elective programs decreased across all school types. Some programs, like the JROTC-CS program with which we are involved [11], often falls within the elective or after school program genre. The lack of student access to JROTC instructors may minimize the effect of any training they received in providing encouragement for students to participate in CS education experiences. This may further the ability for students to learn, since, like formal CS curriculum, hands-on learning for authenticity is important to these programs. We recommend researchers consider how informal learning may also be impacting knowledge gains and noncognitive factors like self-efficacy.
6.3 Limitations
We collected this data from practitioners, and it represents their perspectives. Perspectives can provide clues to what is happening across the CAPE components. Corroborating this data with actual data from students (e.g., actual grades, actual homework completion rates, actual number of students taking additional courses) is still needed. Further, the margin of error on responses sits at 10%, which means that the tests for significance (greater than 25%) on the percentages could potentially lead to a more likely range of 15–35%. This should be taken into consideration when interpreting the results.
With respect to instrument reliability, although we used Cronbach’s alpha as an average measure of internal consistency, we did not conduct a factor analysis on this survey. A factor analysis would provide further information on the validity of the quantitative survey we created for this study [42].
Survey responses from the first survey offered robust data, but primarily from people who identified themselves as White and who were active in CS networks. This instrument, therefore, should be examined more thoroughly to ensure it captures perspectives of other subgroups. The majority (86%) of responses were also affiliated with a CS network, meaning that we lack representation from schools that are not part of a network that is designed to grow and strengthen CS education.
The view of schools serving a majority of students from low income households is based upon the use of reported percentages of students participating in the school’s free or reduced lunch program. Studies have show that this is an imperfect measure of community wealth. Despite this imperfection, the US Department of Education states that “the free and reduced price percentage is useful to researchers from an analytic perspective.” [33]
If we found no significant differences for an item between schools serving students from low-income and higher income families, this does not mean that these schools are operating at an equal level. We did not attempt to provide a baseline for comparing their actual capacity prior to COVID-19. Based on prior literature, we know that schools that serve students from lower-income families offer fewer, if any, CS courses [6].
This study was conducted prior to the receipt of CARES funding, the U.S. government’s Elementary and Secondary School Emergency Relief Fund for the pandemic, that was signed into law on March 21, 2021. Therefore, CARES funding did not affect this study; however, it may affect future studies.
7 Conclusion AND Future Work
Overall, the shifting landscape of what was possible heavily relied on a high school’s commitments to continue with the same number of CS courses and extracurricular activities that would have been offered had the pandemic not occurred. Students and parents made critical choices around course enrollment, perhaps prioritizing “core” academic subjects and family obligations while under the stress of COVID-19. While schools trend to one type of student population, no school is homogeneous. As one participant mentioned in the first survey, student experiences were “wildly different”, and some of our data bears this wildness out across schools.
This work will continue to inform the research team as they seek to evaluate school-based interventions to broaden participation in computer science education. While we do not anticipate continuing to probe the findings in this paper specifically, they will inform larger studies and we hope they will be useful to the larger community in reference to any work conducted from March 2020 until school conditions in the US return to a more steady state.
Acknowledgments
Suppressed for review.
Footnotes
1
Indeed, a single teacher’s capacity to teach students is a dynamic part of the “systems-level” capacity. One system has many individual parts, and a single teachers’ professional development or bandwidth can impact students in the classroom and can be indicative of larger challenges.
2
Research Practice Partnership for CS community in the United States.
3
Expanding Computing Education Pathways to have a significant impact on improving and broadening participation in computing education state by state in the U.S.
4
Within the U.S., schools with a low-income student population of 40% or greater are categorized as Title I by the federal government.
5
Free and reduced-lunch is a classification under the U.S. National School Lunch Program in which students receive either free or reduced lunch costs to their families proximity to the poverty level.
6
This study was conducted prior to the receipt of CARES funding, the U.S. government’s Elementary and Secondary School Emergency Relief Fund for the pandemic, that was signed into law on March 21, 2021. Therefore, CARES funding did not affect this study.
7
One additional item is grouped into Access in the analysis, with one dependent on whether or not fees are charged. Due to this disparity, we removed the Student Fees question from the group of items. We note that the low alpha for the 5-item scale is occurring due to the high amount of Unsure and Not Applicable for the “conflicting classes” item.
7
One additional item is grouped into Access in the analysis, with one dependent on whether or not fees are charged. Due to this disparity, we removed the Student Fees question from the group of items. We note that the low alpha for the 5-item scale is occurring due to the high amount of Unsure and Not Applicable for the “conflicting classes” item.
7
We carefully chose colors from a color-blind friendly palette. Complementary data is also presented in tables and in the text.
References
[1]
Judith S. K. Achoka and Julius Maiyo. 2008. Horrifying disasters in western Kenya impact on education and national development. (2008).
[2]
Muhammad Adnan and Kainat Anwar. 2020. Online learning amid the COVID-19 pandemic: Students’ perspectives.Online Submission 2, 1 (2020), 45–51.
[3]
Wenjun Cao, Ziwei Fang, Guoqiang Hou, Mei Han, Xinrong Xu, Jiaxin Dong, and Jianzhong Zheng. 2020. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Research 287 (2020), 112934.
[4]
Maria C. Cardenas, Samyd S. Bustos, and Rana Chakraborty. 2020. A ‘parallel pandemic’: The psychosocial burden of COVID-19 in children and adolescents. Acta Paediatrica 109, 11 (2020), 2187–2188.
[5]
Simon Cauchemez, Maria D. Van Kerkhove, Brett N. Archer, Martin Cetron, Benjamin J. Cowling, Peter Grove, Darren Hunt, Mira Kojouharova, Predrag Kon, Kumnuan Ungchusak, et al. 2014. School closures during the 2009 influenza pandemic: National and local experiences. BMC Infectious Diseases 14, 1 (2014), 1–11.
[6]
Code.org. 2021. 2020 State of Computer Science Education: Illuminating Disparities. https://advocacy.code.org/2020_state_of_cs.pdf.
[7]
College Board. 2021. Program Summary Report. https://secure-media.collegeboard.org/digitalServices/pdf/research/2020/Program-Summary-Report-2020.pdf.
[8]
Computing Research Association. 2020. Reports on surveys of computer science faculty and academic units/chairs: COVID-19 impact. CRA Bulletin (2020).
[9]
John W. Creswell. 2014. A Concise Introduction to Mixed Methods Research. SAGE publications.
[10]
Tom Crick, Cathryn Knight, Richard Watermeyer, and Janet Goodall. 2020. The impact of COVID-19 and “Emergency Remote Teaching” on the UK computer science education community. In United Kingdom & Ireland Computing Education Research Conference.31–37.
[11]
CSforALL. 2022. JROTC-CS. https://www.csforall.org/projects_and_programs/jrotc/.
[12]
Thomas A. DeVaney, Sonya C. Carr, and Diane D. Allen. 2009. Impact of Hurricane Katrina on the educational system in Southeast Louisiana: One-year follow-up. Research in the Schools 16, 1 (2009), 32.
[13]
Giorgio Di Pietro, Federico Biagi, Patricia Costa, Zbigniew Karpiński, and Jacopo Mazza. 2020. The Likely Impact of COVID-19 on Education: Reflections Based on the Existing Literature and Recent International Datasets, Vol. 30275. Publications Office of the European Union.
[14]
Emma Dorn, Bryan Hancock, Jimmy Sarakatsannis, and Ellen Viruleg. 2020. COVID-19 and learning loss - Disparities grow and students need help. (2020).
[15]
Emma Dorn, Bryan Hancock, Jimmy Sarakatsannis, and Ellen Viruleg. 2020. COVID-19 and student learning in the United States: The hurt could last a lifetime. (2020).
[16]
Carol Fletcher and Jayce Warner. 2021. CAPE: A framework for assessing equity throughout the computer science education ecosystem. Commun. ACM 64 (2021), 23–35.
[17]
Emma Garcia and Elaine Weiss. 2020. COVID-19 and student performance, equity, and U.S. education policy: Lessons from pre-pandemic research to inform relief, recovery, and rebuilding. (2020).
[18]
Paul A. Harris, Robert Taylor, Brenda L. Minor, Veida Elliott, Michelle Fernandez, Lindsay O’Neal, Laura McLeod, Giovanni Delacqua, Francesco Delacqua, Jacqueline Kirby, et al. 2019. The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics 95 (2019), 103208.
[19]
Paul A. Harris, Robert Taylor, Robert Thielke, Jonathon Payne, Nathaniel Gonzalez, and Jose G. Conde. 2009. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics 42, 2 (2009), 377–381.
[20]
Paul Hill and Jane Hannaway. 2006. The future of public education in New Orleans. After Katrina: Rebuilding opportunity and equity into the “New” New Orleans.Urban Institute (NJ1) (2006).
[21]
Institute of Education Sciences. 2021. IES School Survey Dashboard. https://ies.ed.gov/schoolsurvey/#.
[22]
Nataliya V. Ivankova, John W. Creswell, and Sheldon L. Stick. 2006. Using mixed-methods sequential explanatory design: From theory to practice. Field Methods 18, 1 (2006), 3–20.
[23]
Megan Kuhfeld, James Soland, Beth Tarasawa, Angela Johnson, Erik Ruzek, and Jing Liu. 2020. Projecting the potential impact of COVID-19 school closures on academic achievement. Educational Researcher 49, 8 (2020), 549–565.
[24]
John Lai and Nicole O. Widmar. 2021. Revisiting the digital divide in the COVID-19 Era. Applied Economic Perspectives and Policy 43, 1 (2021), 458–464.
[25]
Jihyun Lee and Valerie J. Shute. 2010. Personal and social-contextual factors in K–12 academic performance: An integrative perspective on student learning. Educational Psychologist 45, 3 (2010), 185–202.
[26]
Jane Margolis, Rachel Estrella, Joanna Goode, Jennifer Jellison Holme, and Kim Nao. 2017. Stuck in the Shallow End: Education, Race, and Computing. MIT Press.
[27]
Alexis Martin, Sonia Koshy, Laura Hinton, Allison Scott, Bryan Twarek, and Kalisha Davis. 2020. Teacher perspectives on COVID-19’s impact on K-12 computer science instruction. (2020).
[28]
Andrew McDonald and Laura K. Dillon. 2021. Virtual Outreach: Lessons from a coding club’s response to COVID-19. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 418–424.
[29]
Paul Mihas and Odum Institute. 2019. Learn to Use an Exploratory Sequential Mixed Method Design for Instrument Development. SAGE Publications, Limited.
[30]
Brenda Mitchell, Linda Stelly, and Theresa Perry. 2006. The plight of the education systems–post Hurricane Katrina: An interview with Dr. Brenda Mitchell and Dr. Linda Stelly. The High School Journal 90, 2 (2006), 16–22.
[31]
Catherine Mooney and Brett A. Becker. 2021. Investigating the impact of the COVID-19 pandemic on computing students’ sense of belonging. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 612–618.
[32]
Chipo Mudavanhu. 2014. The impact of flood disasters on child education in Muzarabani District, Zimbabwe. (2014).
[33]
National Center for Education Statistics, US Department of Education. 2015. Free or Reduced Price Lunch: A Proxy for Poverty?https://nces.ed.gov/blogs/nces/post/free-or-reduced-price-lunch-a-proxy-for-poverty.
[34]
Fran H. Norris, Matthew J. Friedman, and Patricia J. Watson. 2002. 60,000 disaster victims speak: Part II. Summary and implications of the disaster mental health research. Psychiatry: Interpersonal and Biological Processes 65, 3 (2002), 240–260.
[35]
Fran H. Norris, Matthew J. Friedman, Patricia J. Watson, Christopher M. Byrne, Eolia Diaz, and Krzysztof Kaniasty. 2002. 60,000 disaster victims speak: Part I. An empirical review of the empirical literature, 1981–2001. Psychiatry: Interpersonal and Biological Processes 65, 3 (2002), 207–239.
[36]
Edeh Michael Onyema, Nwafor Chika Eucheria, Faith Ayobamidele Obafemi, Shuvro Sen, Fyneface Grace Atonye, Aabha Sharma, and Alhuseen Omar Alsayed. 2020. Impact of Coronavirus pandemic on education. Journal of Education and Practice 11, 13 (2020), 108–121.
[37]
Oluwatoyin Mary Oyinloye. 2020. Possible impact of COVID-19 on senior secondary school students’ performance in science education in Nigeria. Journal of Pedagogical Sociology and Psychology 2, 2 (2020), 80–85.
[38]
Andreas Schleicher. 2020. The impact of Covid-19 on education insights from education at a glance 2020. Retrieved from oecd.org website: https://www.oecd.org/education/the-impact-of-covid-19-on-education-insights-education-at-a-glance-2020.pdf (2020).
[39]
Dina Simmons. 2020. Why COVID-19 is our equity check. Educational Leadership Special Report - A new reality: Getting remote learning right. Educational Leadership 77 (2020), 51–53.
[40]
Edgar John Sintema. 2020. Effect of COVID-19 on the performance of grade 12 students: Implications for STEM education. Eurasia Journal of Mathematics, Science and Technology Education 16, 7 (2020), em1851.
[41]
Eva Skuratowicz, Maggie Vanderberg, Eping E. Hung, Gladys Krause, Dominique Bradley, and Joseph P. Wilson. 2021. I felt like we were actually going somewhere: Adapting summer professional development for elementary teachers to a virtual experience during COVID-19. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 739–745.
[42]
Keith S. Taber. 2018. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education 48, 6 (2018), 1273–1296.
[43]
Bishakha Upadhyaya, Monica M. McGill, and Adrienne Decker. 2020. A longitudinal analysis of K-12 computing education research in the United States: Implications and recommendations for change. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 605–611.
[44]
U.S. Bureau of Labor Statistics. 2021. Occupational Outlook Handbook: Teachers. https://www.bls.gov/ooh/education-training-and-library/high-school-teachers.htm.
[45]
U.S. National Center for Education Statistics. 2021. Digest of Education Statistics. https://nces.ed.gov/programs/digest/d19/tables/dt19_105.50.asp.
[46]
U.S. News and World Report. 2020. Disconnected and Disadvantaged: Schools Race to Give Students Access. https://www.usnews.com/news/education-news/articles/2020-04-01/schools-rush-to-get-students-internet-access-during-coronavirus-pandemic.
[47]
U.S. News and World Report. 2020. How COVID-19 is Upending Extracurriculars. https://www.usnews.com/education/best-colleges/articles/how-coronavirus-is-upending-high-school-extracurricular-activities.
[48]
Herman van de Werfhorst, Emma Kessenich, and Sara Geven. 2020. The digital divide in online education. Inequality in digital preparedness of students and schools before the start of the COVID-19 pandemic. (2020).
[49]
World Bank. 2020. The COVID-19 pandemic: Shocks to education and policy responses. (2020).
[50]
Jeffrey Xavier, Alan Peterfreund, Rebecca Zarch, W. Richards Adrion, Renee Fall, Mark Guzdial, Barbara Ericson, Sarah Dunton, and Tom Mcklin. 2019. Fostering state-level change in CS education: The expanding computing education pathways alliance. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 1254.
[51]
Jignasu Yagnik et al. 2014. PSPP a free and open source tool for data analysis. Voice Res. 2 (2014), 73–76.
[52]
Wei Yan, Ruohan Liu, Maya Israel, Heather Sherwood, Cheri Fancsali, and Melissa Pierce. 2020. School-wide integration of computational thinking into elementary schools: A cross-case study. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 1325–1325.
Index Terms
- Practitioner Perspectives on COVID-19’s Impact on Computer Science Education Among High Schools Serving Students from Lower and Higher Income Families
Recommendations
A Framework of Factors that Influence Academic Achievement in Computer Science within Capacity, Access, Participation and Experience
ICER '23: Proceedings of the 2023 ACM Conference on International Computing Education Research - Volume 2Motivation. There are countless factors that contribute to academic achievement [3, 6, 7, 8]. For K-12 computer science (CS) education, these building blocks of student success are broadly captured in the four components of the CAPE framework, which ...
A Landscape Study of Computer Science Education in NYC: Early Findings and Implications for Policy and Practice
SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science EducationNYC's Computer Science for All (CS4All) is a 10-year, districtwide initiative aimed at providing high-quality computer science (CS) education to all NYC public school students. It aspires to greatly increase the number of students, teachers, and schools ...
AP Computer Science Principles' Impact on the Landscape of High School Computer Science using Maryland as a Model
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science EducationThe introduction of the Advanced Placement (AP) Computer Science Principles course changed the high school computer science (CS) landscape in the United States. For the first time an AP course was designed to broaden participation in computing and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 December 2022
Online AM: 16 August 2022
Accepted: 29 July 2022
Revised: 29 April 2022
Received: 30 April 2021
Published in TOCE Volume 23, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
- Refereed
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 1,708Total Downloads
- Downloads (Last 12 months)954
- Downloads (Last 6 weeks)66
Reflects downloads up to 19 Dec 2024
Other Metrics
Citations
Cited By
View allView Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in