Figures
Abstract
The current worldwide pandemic has significantly increased the need for online learning platforms, hence presenting difficulty in choosing appropriate course materials from the vast online educational resources due to user knowledge frameworks variations. This paper presents a novel course recommendation system called the Deep Learning-based Course Recommendation System (DLCRS). The DLCRS combines a hybrid Sequential GRU+adam optimizer with collaborative filtering techniques to offer accurate and learner-centric course suggestions. The proposed approach integrates modules for learner feature extraction and course feature extraction that is performed using (Embeddings from Language Models) ELMO word embedding technique in order to gain a thorough understanding of learner and course profiles and feedback. In order to evaluate the efficacy of the proposed DLCRS, several extensive experiments were carried out utilizing authentic datasets sourced from a reputable public organization. The results indicate a notable area under the receiver operating characteristic curve (AUC) score of 89.62%, which exceeds the performance of similar advanced course recommendation systems. The experimental findings support the viability of the DLCRS, as seen by a significant hit ratio of 0.88, indicating high accuracy in its suggestions.
Citation: Tahir S, Hafeez Y, Humayun M, Ahmad F, Khan M, Shaheen M (2024) Harnessing hybrid deep learning approach for personalized retrieval in e-learning. PLoS ONE 19(11): e0308607. https://doi.org/10.1371/journal.pone.0308607
Editor: Muhammad Bilal, University of Florida, UNITED STATES OF AMERICA
Received: May 23, 2024; Accepted: July 25, 2024; Published: November 13, 2024
Copyright: © 2024 Tahir et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: "All relevant data for this study are publicly available from the Github repository (https://github.com/sidratahir/E-learning) and within the Supporting Information files."
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Traditional education methods are continuously transforming due to the rapid development of the Internet and online educational opportunities. Students and instructors are now content with the constrained traditional curriculum and would instead select courses of interest through other online education platforms [1, 2]. These platforms provide a range of benefits, including access to high-quality, specialized educational courses, comprehensive curricula with clear outlines, and supplementary tasks. Moreover, they give free courses, plentiful learning materials, and flexible learning options. Nevertheless, the rapid proliferation of platforms and courses results in a phenomenon known as "information overload, [3, 4] when learners become susceptible to confusion and decision paralysis when faced with an extensive array of options. This makes it difficult for learners to advance in their education [5, 6]. As a result, it is critical to assist consumers in swiftly selecting the correct course that interests them. Although the courses are categorized when learners utilize the online education platform to study new courses, among numerous courses, swiftly identifying engaging and personalized beneficial courses is critical.
One significance of educational recommendation systems is that they can help improve student’s learning experience and outcomes through learners’ personalized and relevant recommendations for course material [7, 8]. By tailoring course material to each student’s individual needs, interests, and learning styles, recommendation systems can help students stay engaged and motivated, leading to better retention of information and improved academic performance [9, 10]. Additionally, educational recommendation systems can help address the challenge of information overload, as they can filter and prioritize relevant course material, saving students time and effort in finding the resources they need to succeed [11–15].
Deep learning models are a potential way to solve several essential problems in context-aware course retrieval and recommendation systems. These models can make learning much better for students [16]. Deep learning models could solve many issues in context-aware course retrieval or suggestion systems. Classical recommendation systems often fall short when it comes to personalization because they don’t consider each student’s different hobbies and learning styles. Deep learning models can solve this problem because they can use a lot of data to make models that are unique for each learner [17–20]. One of the most significant issues with making sound recommendation systems is that there isn’t enough data, especially when there are a lot of possible things to offer, like educational course materials [21, 22]. Deep learning models can solve this problem by combining a more extensive range of data sources, such as demographic information, activity on social networks, and even biometric data, to make more accurate models. As students learn more, their needs and interests may change. This shows how important it is for recommendation systems to be able to adjust and update themselves in real time [23–25]. This problem can be solved using learner data on how they interact with course materials and deep learning models [26, 27]. This allows for continuous improvement and refining of suggestions [27].
Consequently, this study introduces a unique framework known as the Deep Learning-based Course Recommender System (DLCRS). This system utilizes sequential Gated Recurrent Unit (GRU) to extract learners’ actions and course properties at a comprehensive level from the provided data. The DLCRS framework can assist learners in finding their desired course contents, Addressing sophisticated and multi-dimensional sparsity challenges, and extracting more feature based information than the traditional machine learning techniques [28–30]. Adam optimizer adds value to the accuracy of retrieved results. Learner Feature Extractor (LFE) module collects learner behaviour features whereas Course Feature Extractor (CFE) module that captures course attribute feature information. LFE and CFE are integrated into the derived interaction characteristics in DLCRS. We carried out an experiment with a dataset taken from the real-world educational repository to evaluate the DLCRS framework performed and compared the results with conventional recommendation methods. The experimental findings showed that in terms of recommendation performance, the suggested DLCRS framework beats other previously practiced strategies. The following are the study’s primary contributions.
An exhaustive effort is made to analyze the state-of-the-art and existing Content Recommendation Systems (CRS) that use deep learning and machine learning prediction models in the online education context. This paper presents a specialized leaner feature extraction LFE and course feature extraction CFE module particularly which are adaptive and modular, allowing them to provide input to sequential GRU+adam based control layer for correct prediction and Collaborative Filtering (CF) based recommendation engine [31] for efficient learner-centric recommendation. The paper conducts experiments using real-world educational datasets to illustrate the efficacy of the DLCRS framework over the traditional recommendation techniques. Using a real-world dataset, we also investigated the proposed DLCRS accuracy and Hit Ratio (HR) in the online education venue.
The rest of this study is structured as follows: Section 2 delves into related research on Conventional Recommender Systems (CRS) and different deep-learning approaches in recommender systems. Section 3 delves deeply into the DLCRS framework and its layers. Section 4 provides the experimental evaluation and detailed outcomes of this investigation. Section 5 discusses the results of different experiments, while section 6 concludes the study, discusses its shortcomings, and suggests future research.
2. Related works
This section presents recommender systems based on deep learning that prioritize context-based recommendations. Context-aware recommender systems (CARS), which include contextual information in recommender systems, have arisen as a controversial subject in the industry. The deep learning approach may successfully integrate context information into recommender systems in many complicated recommendation settings.
A multimodal framework of CRS using a deep learning approach is discussed in Xinwei [32]. They proposed a course recommendation model having an Attention Mechanism and Long- and Short-Term Memory (LSTM) to overcome the issue of course recommendation. User demographic data and course rating information were combined to generate learner preferences. They conducted extensive and rigorous trials on actual datasets, and the results indicated 79% for AUC, which is promising accuracy to provide learners more accurate recommendation outcomes while course condition.
In another work, Hazrati [33] introduced a Collaborative Filtering (CF) technique based on restricted Boltzmann machines. They used deep learning in a recommendation system to find hidden qualities of persons and courses. They concluded that RBM’s CF approach has various disadvantages, including a lengthy training period and a broad range of weight values linking the hidden and visible layers, making it challenging to utilize in practical applications.
Another research [34] presented a CRS Based on DL for boosting the efficiency of user learning. Their work tried to overcome traditional recommendation challenges by introducing a novel DL-based CRS called DECOR. The proposed work gathered detailed user behaviors and course attribute data. The proposed model dealt with problems with high-dimensional data sparsity, reduced the amount of excessive information to process, and extracted information about features of high quality. By doing various tests with datasets taken from the real world, researchers evaluated the performance of their model and compared it with conventional RS. The study’s findings indicated that their model outperforms previous strategies in terms of recommendation performance.
Moreover, You discussed [35] coupled RNN with an innovative HierTCN to enable a hierarchical understanding of user inclinations and to match findings to current data in order to depict the historical effect on more recent user selections. Wan [17] proposed using learning objects and conducted their research grounded in the self-organization theory and a learner model that employed a knowledge-based approach. The system successfully addressed the cold start issue, albeit with a delay due to the presence of multiple layers of algorithms. In order to boost the RS, Trifa [36] included conceptual correlation and an adaptive key-value memorization network into a knowledge-tracing agent. Knowledge structures and recommended study material linkages were developed by Nitchot [37] with the assistance of logic, ontology, and computing.
Wu [38] used Context-aware period recommendation using RNN. Before integrating these attributes into a session-based RNN recommendation model with three layers of perceptron, the approach initially transformed contextual information into low-dimensional actual vector data. Furthermore, they integrated the functional extension module, which relies on extensive sequences. The researchers conducted comprehensive examinations on two publicly available datasets. The experimental results demonstrated that the model outperformed other state-of-the-art models regarding suggestion performance.
Sadia [39] mentioned that the critical role of the recommendation system in offering quality training resources and the deficiency of online support from services is the main reason for many difficulties. Implementing a system that provides intelligent course recommendations, considering several viewpoints, is necessary to enhance students’ competencies and knowledge. Their work offered an architecture having virtual agents to make contextual recommendations based on user preferences to help academia find relevant course materials. The experimental and statistical findings suggested that their approach increased user learning capabilities and simplified course selection based on learners’ preferences and interests.
Shanshan presented an enhanced hybrid ontology-based technique for suggesting online learning resources [46], that incorporated CF and Sequential Pattern Mining (SPM) algorithms. To avoid data sparsity and cold start difficulties, ontology was effectively leveraged for knowledge representation. The analysis of learners’ sequential access patterns in history contributed to the formulation of recommendations that align more closely with educational policies and regulations. The experimental findings demonstrated that their enhanced hybrid approach for educational material recommendations exhibited superior performance and recommendation quality compared to other pertinent techniques.
In another paper, Bhaskaran [18] employed an upgraded vector space recommender that followed the learner’s requirements, preferences and awareness level. The different ways students learn From the server blogs were put into groups and collected. After adding more preprocessing steps to make a better list of suggestions, the similarity was determined with better content-based filtering. The changed cosine similarity content was used to decide how the results were put in order. The CF method suggested putting all of the busy learners into one cluster. The proposed framework was tested using Machine Learning (ML) benchmark cases, including music, business, movie, food, book, healthcare, and Open University courses. When compared to existing, well-known methods, the simulation results of the suggested model showed better performance, precision, recall, accuracy, Mean Absolute Error (MAE), and ranking score. Some other relevant research contributions are summarized in Table 1.
From the above discussion, it has been observed that several review studies based on traditional and hybrid recommendation methodologies have been published on e-learning recommender systems. These assessments explore several aspects of recommendation models, including conventional approaches, ontology-based tactics, algorithms that use machine learning, deep learning-inspired approaches, and a comparison of different recommendation strategies in the context of e-learning systems. However, our search yielded no academic instances of context-aware e-learning course recommender systems incorporating learner and course feature extraction at the input stage. This work aims to bridge the existing research gap by integrating recent studies on contextual recommenders within the e-learning domain.
3. Methods
This section describes the proposed Deep Learning-based Course Recommender System DLCRS, built on the deep learning framework and associated dataset used in this research work. This framework classifies and recommends courses based on various course and learner information. The learner’s actions while looking for and utilizing course materials are recorded and evaluated in order to provide learner-centric recommendations. The DLCRS comprises four major components: data preparation, feature embedding, deep learning course classification, and CF-based recommendation which are added in three layers. The entire process for the proposed DLCRS can be seen in Fig 1. The input layer prepares the dataset and keeps a repository of user preferences, user learning goals, and user and course logs. ELMo based word embedding is created and vector is given to control layer where sequential GRU comes in action. The final output is given to a collaborative filtering module that recommends the learning content based on the collaborative score of the end user.
3.1 Dataset collection
The collection and selection of data are fundamental components of any experimental study. In our research, we have utilized a benchmark dataset employed in Learning Management System (LMS) of a credible public sector university [47]. The dataset is only in the English language. The dataset included a wide range of learner profiles, course offerings, and interaction data, which offered a comprehensive and representative sample for our testing. The dataset complies with existing regulations regarding privacy and access of the LMS. The university had stringent measures and safeguards to ensure the privacy and confidentiality of learner data obtained through online learning platforms. These procedures included techniques to remove identifying information from data, mechanisms to obtain user consent, and compliance with relevant data protection rules. It was ensured that learner data is used for research purposes ethically and responsibly.
During the evaluation of the DLCRS, a significant difficulty was identified regarding the accessibility and accuracy of the datasets used to train and test the recommendation system. Although we obtained genuine dataset from a well-regarded public sector university, it is crucial to recognize the possible biases, noise, and incompleteness that are inherent in real-world data. In order to address this difficulty, we implemented thorough data preprocessing and cleaning techniques to improve the reliability and robustness of the experimental results explained in following sections. This study was approved by the University’s IRB. Informed consents were obtained from all participants online (IRB available in supporting documents). As no minors were involved in this study, parental or guardian consent was not required. In addition, we followed ethical guidelines to verify that our research complied with ethical standards and legislative regulations regarding the use of data from human subjects.
3.2 Input layer
The e-learning site gets its information from two sources: the student feature extractor (LFE) module and the course feature extractor (CFE) module. The user profile details, learning goals, and learning choices comprise most learner data. The course metadata and course characteristics make up most of the course material. Fig 2 shows the main progression of data collection.
The ELMo (Embeddings from Language Models) approach builds word representations based on the output of a bidirectional language model [48]. The ELMo embedding for a particular token is computed as a linear combination of all layers’ hidden states in the bidirectional language model, with the linear combination’s weights learned during training:
(1)
Where:
ELMoi = the ELMo embedding of ith token in the input sequence
L = bidirectional language model layers count
lin = nth layer hidden state of ith token
wn = scalar weight assigned to the nth layer, added at the training phase
δ = scalar parameter that gauges the overall scale of the ELMo vector
3.3 Control layer
The prediction layer employs GRU to identify short- and long-term dependencies in input text sequences [49]. Although it is possible to manage a series of text data with a single GRU layer, adding multiple GRU layers can improve model performance. The primary objective of employing a sequence of GRU layers is to enable the model to learn increasingly complex text input patterns [50, 51]. A text input is initially processed by an embedding layer, which converts each word into a high-dimensional vector. When encoded text is passed through a series of GRU layers, the material is sequentially analyzed and relationships between words and phrases are recorded. The GRU offered a simplified version of the LSTM memory cell with comparable efficacy but faster calculation speed. Each successive GRU layer acquires a distinct level of abstraction.
The initial layer may learn straightforward textual patterns, such as the occurrence of particular words and phrases. The second layer can then learn more complex patterns by combining the fundamental patterns from the first layer. This process can be replicated with multiple layers, allowing the model to discover more complicated and abstract representations of the text data. As an initial vector sequence, this study represents all characteristics of sample turnover.
where n indicated to the number of the features in profile.
(2)
(3)
where σ expresses the sigmoid function. Wr,Wz,Pr,Pz are weight parameters and Sh and Sr are bias parameters. Then, we integrated the reset gate as defined:
(4)
where Wxh,Phh are weight parameters, Sh is the bias, and the symbol ⊙ is the element wise product, and rt determines procedure to join the new input feature with the previous memory. Here tanh is used as activation function. Finally, gate ut is updated. This determined the extent to which the new hidden state matches the old state ht−1 versus how much it resembled the new candidate state
. The update gate ut used elementwise convex combinations of
and ht−1. This provided the final update equation for the GRU:
(5)
Where ut determines nth number of previous memories are joined with the current state, and specifies the candidate state of ht.
The output of the final GRU layer is fed into a fully connected (FC) layer, which learns a high-level representation of the courses. Finally, the output of the FC layer is fed into a softmax layer, which outputs a probability distribution over the possible course classes. Adam is renowned for its capacity for adjustable learning rates. It separately adjusts the learning rate for each parameter based on the historical gradients. This adaptability can assist in overcoming difficulties related to manually fine-tuning learning rates, which can be tiresome and ineffective. Adam is explained as follows.
Algorithm: Adam to optimize results
Set Learning rate: α = 0.001
β1 = 0.9(exponential decay rate)
β2 =0.89 (exponential decay rate)
ε = 1e−7
t(current iteration step 1 to 5)
START
1
2
3
4
5
6
7
8
9
where, θ represents the neural network’s parameters (weights and biases). α is the learning rate, which controls the step size during parameter updates. β1 and β2 are the exponential decay rates for the first and second moment estimates. ε is a small constant to prevent division by zero.
t is the current iteration step or timestep. g represents the gradient of the loss concerning the parameters. m is the first moment estimate (the moving average of gradients). v is the second moment estimate (the moving average of squared gradients). m_hat and v_hat are bias-corrected moment estimates.
The parameters θ are updated using the corrected moment estimates, scaled by the learning rate and the square root of the corrected second moment estimate. The Adam optimizer adapts the learning rates for each parameter based on the historical gradient information, making it suitable for a topic classification accuracy.
3.4 Output layer
Once the expected course content is collected from the prediction layer, it is delivered to the CF-based recommendation layer. CF is a crucial component of the DLCRS since it leverages the combined expertise of users to provide personalized course recommendations. The DLCRS utilizes a user-based collaborative filtering method, which involves identifying users who have similar tastes and recommending courses that have received positive evaluations from these similar users. This integration encompasses multiple essential stages: 1) Computing user similarity by analyzing past interactions and preferences, 2) forming a user neighborhood consisting of users with similar preferences, 3) combining preferences within the neighborhood to determine popular courses, and 4) incorporating these combined preferences into the deep learning model, along with additional contextual information such as course descriptions and user profiles. By integrating collaborative filtering approaches, the DLCRS architecture is enhanced by supplementing the capabilities of the deep learning model. This improves accuracy and relevance in providing personalized course recommendations for individual users. The DLCRS efficiently integrates collaborative filtering with deep learning to improve recommendation accuracy by harnessing the characteristics of both approaches.
In order to obtain learner-centric course material, it is necessary to get information on the same subjects from other relevant learners. CF is beneficial in collaborative learning environments when many learners contribute comments on a single course C [9].
The active learner rating for any Course C might alter the ranking list of topics t for this purpose. These details are obtained from the Course rating logs that are accessible through the CFE module. To address this concern, the proposed DLCRS framework is enhanced further by incorporating CF having Rating factor ℛ, in which retrieved topic is rated by learner on a scale of 1 to 5 as shown in Fig 3. Eq (6) shows the highly rated topic t with user rating impact.
Where indicated predicted score of topic t given by active learner ℓ having
learning preference.
is Pearson correlation for
and leaner ℓ.
The core principle of CF users is that U = u1, u2,…un rate or behave similarly concerning the subject. Topics t = t1, t2,…, tn in an e-learning environment where participants rated or behaved similarly on document D’s topic t. A collection of user preferences for LOs was used by CF techniques to forecast what topics a new user might be interested in learning more about. Ratings may be expressed explicitly, as with a 1–5 scale. For instance, the list of users and their preferred or least preferred topics in Table 2 is transformed into a user-item rating matrix, as shown in Table 3, where ui is the current user who gave recommendations. Participants did not indicate their preferences for particular matrix elements.
In this context, individuals with comparable topics and shared areas of interest are typically catered to. Every user u is treated as a vector in the multidimensional space of the topic t, and the "Pearson correlation" is used to calculate the relationships between the active user and the remainder of the users. Following the discovery of the k most comparable users, "the related rows in the user-item matrix R are aggregated to identify a collection of things, C, that the group has preserved together with their frequency" (in this case it is course topics). User-based CF methods then propose the top-N topic with the highest score in T that the user has not stored. As shown in Table 4. Top-N recommendations from user systems are subject to scalability and real-time performance limitations.
The r(ℓ,x) is score of topic t by leaner ℓ on the Likert scale of 1 to 5. The existing for topics t recommendation is forecasted with the help of sum of comparative rating in time T, shown in Eq (7). These details are obtained from the Learner rating logs that are accessible through the LFE module.
The new learning preference is determined by taking the previous
and the current
and averaging them.
Pearson correlation measures the similarity ρ(ℓ1, ℓ2) between learners, ℓ1 and ℓ2, or ρ(t1,t2) between two topics i and j, on the basis of correlation. The Pearson correlation coefficient assesses the linear relationship between two variables as shown in Eq (8).
(8)
where “i ∈ t summations are over the topics that both the learners ℓ 1 and ℓ 2 have rated” while “rl is the average rating of the co-rated items of the ith user”. Afterwards, topic relationship and similarity among learners is calculated, “the set of learners ℓ ∈ L who rated both topics ti and tj”, the “Pearson Correlation” from Eq (9).
The ranked topics are then transferred to the learner dashboard as an output. By employing the cosine similarity, the degree of similarity that exists between ti and tj in terms of their rating similarity is calculated using Eq (10).
the rating given to t sub i. by the rl,i is rating given to ti by learner ℓ rl is mean rating of all rating provided by leaner ℓ.
4. Experimental evaluation
To demonstrate the efficacy of the proposed recommendation framework, we conducted a number of experiments and assessed our DLCRS framework using an LMS-based real-world dataset. The primary drive of the experiment was to evaluate and compare the performance, improvement in quality, and examine precision of the predictions made by the proposed DLCRS.
4.1 Experimental setup
Learner data is gathered for the purpose of assessment through the LMS, and this data is received through the interaction of learners with the LMS. This research makes use of a real-world dataset, which consists of 380 learners’ data, 2700 learner ratings, and 468 annotated documents of course content pertaining to the subject of software engineering. Learner data is gathered from the students attending a university that is part of the public sector. The data from undergraduate and postgraduate students who are currently enrolled in Computer Science and Software Engineering programs are contained in the learner dataset. The information for the dataset was gathered over the course of one year, notably during the COVID-19 epidemic. The code for the proposed system was implemented using the TensorFlow-Keras libraries in Python Jupiter Notebook on the Google Colab platform. The development environment utilized an Intel(R) Core(TM) i5-6200U CPU running at a frequency of 2.30GHz with 8GB of RAM. Additionally, a GPU with 12GB of memory was employed in the development process. In the conducted studies, the ratio of training data to testing data was set at 80% to 20%. The Adam optimizer was programmed using the subsequent hyper-parameters: The experiment was conducted over a total of 200 epochs, with a batch size of 64. The loss function used for this experiment was "cross entropy".
4.2 Evaluation parameters
The Area Under Curve (AUC) represents the area under the ROC curve, and it is the probability that a relevant course ranks higher than an irrelevant course based on its rating. Assume the suggested course list comprises C0 relevant course samples and C1 irrelevant course samples, with k relevant samples projected to be greater than k irrelevant samples; then, the AUC calculation may be described as follows:
(11)
The hit rate [35] is represented by the Hit Ratio (HR), which is the proportion of learners who have k suitable suggestions in the suggested course list. For each learner’s Top-N list, while the denominator T represents all test sets. The numerator NumberHits@i is the total of the number of test sets i, the HR is calculated as:
(12)
Recall [42] relates to how much of the recommended course’s relevant course elements were accurately predicted. Recall can be written as:
(13)
Normalized Discounted Cumulative Gain (ℕDCG) [52] is frequently used to evaluate the recommended ranking outcomes. NDCG can be written as:
(14)
(15)
NCGu#i indicates the discounted cumulative of the learner’s accurate list; DCGu@i is the score of the leaner’s accurate list and CℕDCG@i signifies the average of each learner.
5. Results and discussion
For the purpose of assessing how effective the proposed DLCRS framework was conducted in controlled environment. Learners who studied the available courses on the application accomplished this activity by browsing and clicking on the specific courses they wanted by searching for them using keywords such as the course name and suggested courses. They compiled a list of the classes they tried to access and ranked them. The platform kept a record of the various operational activities of learners and gathered explicit and implicit response data from learners in order to gain a deeper understanding of the watching patterns of learners by looking at logs. When learners accessed the material for the course, they additionally downloaded it and stored it in the appropriate directory. Through this research, we effectively improved the performance of DLCRS as a result of fusing the modal information from LFE and CFE. Additionally, in order to effectively enhance the performance of the recommendation, GRU successfully explored the important contents contained within the various contextual information as well as the progressive information that exists between courses and learners. The DLCRS framework that was presented in this paper achieved better course recommendation results, with the AUC score reaching 89.62% and the hit rate reaching 88% topic wise and 94% learner wise; indicating that the proposed DLCRS framework is able to improve desired course recommendation accuracy effectively. The experimental results show that the proposed DLCRS framework was able to achieve enhanced results as shown in Fig 4. An AUC score of 89.62% indicates that the DLCRS framework has a high ability to distinguish between positive and negative outcomes. In other words, it can accurately suggest courses that match learners’ interests and preferences. A higher AUC value signifies that the DLCRS has a superior capability to accurately suggest appropriate courses to users, taking into account their preferences and interactions. This suggests that the DLCRS is proficient in precisely capturing and utilizing the implicit and explicit reactions of learners, as well as their browsing and interaction practices, to provide customized course recommendations.
In order to assess the usefulness of the suggested course materials, we conducted an analysis and experimentation to test the impact of various topic feature combinations on the outcomes. Fig 5 depicts a comparison of information hit rates across five topics for a specific course.
The experimental results indicate that t2 has the lowest hit rate, suggesting that it contains the least amount of relevant information. T5 has the maximum HR for the topic, which is 88 percent. When the hit rate of t3 is compared to the HR of t4, it is found that course-related information enhances the accuracy of the recommendation, indicating that t4 contains more course-related information. Thus, it is demonstrated that contextual course topic information improves the accuracy of recommendation. Fig 6 reflects the validation loss and validation accuracy over 10 epochs during training. Training and validation accuracy graphs are essential machine learning tools for model performance and generalization evaluation. These graphs show a model’s accuracy metrics on the training and validation datasets across epochs.
Training accuracy in Fig 6 shows how well the model learns training data patterns. As the model iteratively adjusts its parameters to minimize training dataset mistakes, accuracy increases. Training accuracy may rise substantially if the model learns the dataset well. The validation accuracy graph shows how effectively the model generalizes to unseen data (validation set) during training. It helps determine model overfitting or underfitting. If validation accuracy falls behind training accuracy or stagnates while training accuracy rises, the model is overfitting and memorizing the training data rather than learning its patterns.
The demographic characteristics of learners yield a limited amount of information and have a low success rate in relation to course outcomes. Including learners’ explicit feedback and scoring record in the form of content rating as well as implicit feedback data (browsing activities on the course) may significantly increase user hit rate. The above experimental findings suggest that combining different user information boosts course hit rate and improves course recommendation effect. Fig 7 shows the hit rate of different learner features.
Similarly, the ranking of recommended course contents improved with HR increase. As shown in Fig 8, the HR reached 90% as contextual information increases the information gain. At the same time, the NDCG index of ranked course information increase with HR boost. The increase in recall and NDCG indicate that recommendation of contextual information for the specified topic for learners is improving.
The comparison of performance metrics is carried out using AUC, NDCG, HR, Recall and Precision. The performance evaluation uses metrics namely Precision, F-measure and Recall to assess the precision and efficiency of the proposed DLCRS. Table 5 explains the collective assessment of proposed DLCRS with state of art recommendation systems.
The dataset DS1 [32] has used LSTM and AM based framework and content-based datasets acquired from educational sites were utilized. The AUC is 0.53, the recall is 0.58, HR is 0.79%. Similarly, the dataset DS2 [34] after experimenting with a publicly available data set from Korean LMS. In this research, collaborative neural filtering is employed. 0.56 recall, 0.58 AUC, 0.52 HR is achieved. Moreover, the dataset used in [45] is abbreviated as DS3, acquired dataset from publicly available MOOC utilizing bi- LSTM and CNN approach. Calculated AUC was 0.71, recall was 0.43, NDCG was 0.59, and HR was 0.81. Fig 9 demonstrates the comparison among existing approaches and proposed one.
As shown in Fig 10, DS1 has lower AUC and recall as compared to DLCRS. The proposed recommender system is efficient as its accuracy is higher than DS1. Similarly, DS2 has lower HR and accuracy from proposed dataset. The recall of proposed system is higher than DS2. When compared with DS3, the proposed system outperforms DS3 with higher AUC and recall. Collectively, proposed system is found to be more efficient and productive because of higher HR.
The results of experiments have revealed necessary online learning platform improvements: First, the DLCRS emphasizes personalized course recommendations for individual learners. Online systems must be flexible to meet varied learner preferences and styles. Deep learning and collaborative filtering enable personalized, engaging, and satisfying learning. Second, the DLCRS’s correct course recommendations enhance users’ learning. This shows how advanced recommendation algorithms simplify course material access. Platform recommendation algorithms that consider learner preferences and interactions can boost user happiness and retention. Our DLCRS experiments reveal learner behavior and preferences, enabling data-driven online platform decision-making. Administrators can refine course offerings, identify areas for improvement, and adjust content to user needs by analyzing feedback and performance analytics. Finally, our repeated experimentation emphasizes online platform design’s need for continual improvement. Develop recommendation algorithms, content strategies, and user interfaces using feedback mechanisms and performance indicators to improve effectiveness and user experience. To conclude the discussion, the proposed DLCRS framework has a lot of potential in terms of the recommendation of courses that are user-friendly and efficient due to the speed with which it operates. Learners have the ability to access and make use of these resources by choosing a variety of course resources, lesson plans. Moreover, the retrieved course materials are contextually related to one another. In conclusion, the intuitive user interface of the proposed DLCRS is adaptive at catering the needs of the learner that even a novice would feel at ease using it while accessing course content.
5.1 Implication of the study
The DLCRS paradigm has substantial implications in the realm of recommendation systems, especially in e-learning contexts. Enhanced personalization has facilitated the retrieval of course resources that closely match user preferences, leading to practical applications. By identifying more appropriate course materials, learners can achieve greater educational achievements. Analyzing information acquired from learners’ preferences can assist e-learning platforms in determining course offerings, designing content, and enhancing e-learning platform services. The incorporation of contextual recommendations enhances the effectiveness of collaborative filtering and personalization in e-learning by utilizing learners’ interactions, thereby introducing a fresh perspective on improving the accuracy of e-learning suggestions. Lastly, DLCRS enhances the existing expertise on the application of deep learning techniques in educational recommendation systems.
5.2 Limitations
While the DLCRS technique has shown encouraging outcomes, it is important to acknowledge a few limitations. As learners’ interests and requirements may change over time, this evolution can impact their learning preferences. Currently, the proposed DLCRS does not consider the temporal changes in the learner’s preferences. In addition, both new learners and new course material may encounter the cold start problem, which might impact their recommendation and ranking. To enhance the effectiveness and capability of DLCRS, it is important to solve the critical issue of cold start and temporal changes in future.
6. Conclusion
This study aimed to propose a hybrid deep neural network that integrates GRU sequences and heterogeneous features of learners’ preferences. The objective was to enhance e-learning platforms by enabling collaborative filtering for recommending course content. This is important because both the sheer number of students and the number of online classes are growing at a swift rate. First, we focused on using a word embedding approach based on the ELMo for obtaining a high-dimension vector of features and initialized learners as a comparable group. The top-N probable courses are then generated using sequential GRU, synchronous sequences having heterogeneous profile characteristics. Course topics are recommended to learners via collaborative filtering. Our study aimed to offer context-aware course content to participants based on their learning preferences.
The main contribution of this study was the incorporation of sequential GRU with the integration of diverse features leading to enhanced collaborative filtering. In addition, ELMo word embedding was utilized to enhance feature representation. Furthermore, the DLCRS approach suggests customized LOs, resulting in improved learning resources for learners. The proposed DLCRS surpasses existing personalized e-learning recommendation systems with NDCG and AUC scores of 94% and 89% respectively.
However, several restrictions must be overcome in order for it to progress:
- To solve learners’ cold start issue using deep learning and sentiment analysis
- Combine the method with more advanced and precise clustering algorithms such as bi-LSTM.
Furthermore, future research should look at creating sentiment-based feedback on course development models that use user input and learners’ history sequence data, as well as developing a recommendation model for higher education courses in the multi-context domain. In future studies, we also plan to explore the scalability of the DLCRS by evaluating its performance effectiveness with increasingly large and diverse datasets.
References
- 1. Coman CȚ\^\iru LG, Meseșan-Schmitz L, Stanciu , Bularca MC. Online teaching and learning in higher education during the coronavirus pandemic: Students’ perspective. Sustainability. 2020;12: 10367.
- 2. Yeung MWL, Yau AHY. A thematic analysis of higher education students’ perceptions of online learning in Hong Kong under COVID-19: Challenges, strategies and support. Educ Inf Technol. 2022; 1–28.
- 3. Belabbes MA, Ruthven I, Moshfeghi Y, Rasmussen Pennington D. Information overload: a concept analysis. J Doc. 2023;79: 144–159.
- 4. Sarwar S, Tahir S, Humayun M, Almufareh MF, Jhanjhi NZ, Hamid B. Recommendation of Smart Devices Using Collaborative Filter Approach. 2022 14th International Conference on Mathematics, Actuarial Science, Computer Science and Statistics (MACS). 2022. pp. 1–4.
- 5. Fu S, Li H, Liu Y, Pirkkalainen H, Salo M. Social media overload, exhaustion, and use discontinuance: Examining the effects of information overload, system feature overload, and social overload. Inf Process \& Manag. 2020;57: 102307.
- 6. Al-Kumaim NH, Alhazmi AK, Mohammed F, Gazem NA, Shabbir MS, Fazea Y. Exploring the impact of the COVID-19 pandemic on university students’ learning life: An integrated conceptual motivational model for sustainable and healthy online learning. Sustainability. 2021;13: 2546.
- 7. Jena KK, Bhoi SK, Malik TK, Sahoo KS, Jhanjhi NZ, Bhatia S, et al. E-Learning Course Recommender System Using Collaborative Filtering Models. Electron. 2023;12.
- 8. Safarov F, Kutlimuratov A, Abdusalomov AB, Nasimov R, Cho YI. Deep Learning Recommendations of E-Education Based on Clustering and Sequence. Electron. 2023;12.
- 9. Tahir S, Hafeez Y, Abbas MA, Nawaz A, Hamid B. Smart Learning Objects Retrieval for E-Learning with Contextual Recommendation based on Collaborative Filtering. Educ Inf Technol. 2022; 1–38.
- 10. Zapata A, Menéndez VH, Prieto ME, Romero C. A framework for recommendation in learning object repositories: An example of application in civil engineering. Adv Eng Softw. 2013;56: 1–14.
- 11. Zhu M, Berri S, Zhang K. Effective instructional strategies and technology use in blended learning: A case study. Educ Inf Technol. 2021. pmid:34121908
- 12. De Medio C, Limongelli C, Sciarrone F, Temperini M. MoodleREC: A recommendation system for creating courses using the moodle e-learning platform. Comput Human Behav. 2020;104: 106168.
- 13. Limongelli C, Lombardi M, Marani A, Sciarrone F, Temperini M. A recommendation module to help teachers build courses through the Moodle Learning Management System. New Rev Hypermedia Multimed. 2016;22: 58–82.
- 14. Xu G, Jia G, Shi L, Zhang Z. Personalized Course Recommendation System Fusing with Knowledge Graph and Collaborative Filtering. 2021. pmid:34616447
- 15. Klašnja-Milićević A, Ivanović M, Vesin B, Budimac Z. Enhancing e-learning systems with personalized recommendation based on collaborative tagging techniques. Gasparetti. 2018;48: 1519–1535.
- 16. Abdo AM, Maisara N, Rasid A, Amira N, Mohd H, Nur S, et al. Student ‘ s Performance Based on E-Learning Platform Behaviour using K-means Clustering. 2021;7: 72–78.
- 17. Wan S, Niu Z. An e-learning recommendation approach based on the self-organization of learning resource. Knowledge-Based Syst. 2018;160: 71–87.
- 18. Bhaskaran S, Marappan R. Enhanced personalized recommendation system for machine learning public datasets: generalized modeling, simulation, significant results and analysis. Int J Inf Technol. 2023;15: 1583–1595.
- 19. Guan Z, Bu J, Mei Q, Chen C, Wang C. Personalized tag recommendation using graph-based ranking on multi-type interrelated objects. Proc 32nd Int ACM SIGIR Conf Res Dev Inf Retr—SIGIR ‘09. 2009; 540.
- 20. Al-Quayed F, Humayun M, Tahir S. Towards a Secure Technology-Driven Architecture for Smart Health Insurance Systems: An Empirical Study. Healthcare. 2023. p. 2257. pmid:37628455
- 21. Feng C, Liang J, Song P, Wang Z. A fusion collaborative filtering method for sparse data in recommender systems. Inf Sci (Ny). 2020;521: 365–379.
- 22. Natarajan S, Vairavasundaram S, Natarajan S, Gandomi AH. Resolving data sparsity and cold start problem in collaborative filtering recommender system using linked open data. Expert Syst Appl. 2020;149: 113248.
- 23. Tarus JK, Niu Z, Kalui D. A hybrid recommender system for e-learning based on context awareness and sequential pattern mining. Soft Comput. 2018;22: 2449–2461.
- 24. Tarus J, Niu Z, Khadidja B. E-Learning Recommender System Based on Collaborative Filtering and Ontology. Int J Comput Inf Eng. 2017;11: 400–405.
- 25. Nabizadeh AH, Gonçalves D, Gama S, Jorge J, Rafsanjani HN. Adaptive learning path recommender approach using auxiliary learning objects. Comput Educ. 2020;147: 103777.
- 26. Lin J, Liu C, Li Y, Cui L, Wang R, Lu X, et al. Automatic Knowledge Discovery in Lecturing Videos via Deep Representation. IEEE Access. 2019;7: 33957–33963.
- 27. Vijayalakshmi V, Venkatachalapathy K. Comparison of Predicting Student‘s Performance using Machine Learning Algorithms. Int J Intell Syst Appl. 2019;11: 34–45.
- 28. Gasparetti F, De Medio C, Limongelli C, Sciarrone F, Temperini M. Prerequisites between learning objects: Automatic extraction based on a machine learning approach. Telemat Informatics. 2018;35: 595–610.
- 29. N.P. J, Aruna R. Big data analytics in health care by data mining and classification techniques. ICT Express. 2021.
- 30. Ali T, Asghar S. Multi-label scientific document classification. J Internet Technol. 2018;19: 1707–1716.
- 31. Wang Z, Wang Z. Intelligent recommendation model of tourist places based on collaborative filtering and user preferences on collaborative filtering and user preferences. 2023.
- 32. Ren X, Yang W, Jiang X, Jin G, Yu Y. A Deep Learning Framework for Multimodal Course Recommendation Based on LSTM+Attention. Sustain. 2022;14: 1–14.
- 33. Hazrati N, Shams B, Haratizadeh S. Entity representation for pairwise collaborative ranking using restricted Boltzmann machine. Expert Syst Appl. 2019;116: 161–171.
- 34. Li Q, Kim J. A deep learning-based course recommender system for sustainable development in education. Appl Sci. 2021;11.
- 35. You J, Wang Y, Pal A, Eksombatchai P, Rosenburg C, Leskovec J. Hierarchical temporal convolutional networks for dynamic recommender systems. The world wide web conference. 2019. pp. 2236–2246.
- 36. Trifa A, Hedhili A, Chaari WL. Knowledge tracing with an intelligent agent, in an e-learning platform. Educ Inf Technol. 2019;24: 711–741.
- 37. Nitchot A, Wettayaprasit W, Gilbert L. Personalized learning system for visualizing knowledge structures and recommending study materials links. E-Learning Digit Media. 2019;16: 77–91.
- 38. Wu T, Sun F, Dong J, Wang Z, Li Y. Context-aware session recommendation based on recurrent neural networks. Comput Electr Eng. 2022;100: 107916.
- 39. Ali S, Hafeez Y, Humayun M, Jamail NSM, Aqib M, Nawaz A. Enabling recommendation system architecture in virtualized environment for e-learning. Egypt Informatics J. 2021.
- 40. Wu L. Collaborative Filtering Recommendation Algorithm for MOOC Resources Based on Deep Learning. Complexity. 2021;2021.
- 41. Dwivedi P, Bharadwaj KK. E-Learning recommender system for a group of learners based on the unified learner profile approach. Expert Syst. 2015;32: 264–276.
- 42. Jeevamol J, Renumol VG. An ontology-based hybrid e-learning content recommender system for alleviating the cold-start problem. Educ Inf Technol. 2021;26: 4993–5022.
- 43. Ontology D, Amane M, Aissaoui K, Berrada M. ERSDO: E-learning Recommender System based on. 2022; 7549–7561.
- 44. Joy J, Raj NS, Renumol VG. Ontology-based E-learning Content Recommender System for Addressing the Pure Cold-start Problem. J Data Inf Qual. 2021;13.
- 45. Deng W, Zhu P, Chen H, Yuan T, Wu J. Knowledge-aware sequence modelling with deep learning for online course recommendation. Inf Process \& Manag. 2023;60: 103377.
- 46. Shanshan S, Mingjin G, Lijuan L. An improved hybrid ontology-based approach for online learning resource recommendations. Educ Technol Res Dev. 2021;69: 2637–2661.
- 47.
Arid University. [cited 6 Apr 2024]. Available: http://umsportal.uaar.edu.pk/arid/public/login
- 48. Rodrigues RC, Rodrigues J, de Castro PVQ, da Silva NFF, Soares A. Portuguese language models and word embeddings: evaluating on semantic similarity tasks. Computational Processing of the Portuguese Language: 14th International Conference, PROPOR 2020, Evora, Portugal, March 2–4, 2020, Proceedings 14. 2020. pp. 239–248.
- 49. Lu Y, Liu L, Panneerselvam J, Yuan B, Gu J, Antonopoulos N. A gru-based prediction framework for intelligent resource management at cloud data centres in the age of 5g. IEEE Trans Cogn Commun Netw. 2019;6: 486–498.
- 50. Lin H, Sun Q. Crude oil prices forecasting: an approach of using CEEMDAN-based multi-layer gated recurrent unit networks. Energies. 2020;13: 1543.
- 51. Jung S, Moon J, Park S, Hwang E. An attention-based multilayer GRU model for multistep-ahead short-term load forecasting. Sensors. 2021;21: 1639.
- 52. Ibrahim OAS, Landa-Silva D. An evolutionary strategy with machine learning for learning to rank in information retrieval. Soft Comput. 2018;22: 3171–3185.