Level-Based Learning Algorithm Based on the Difficulty Level of the Test Problem

Abstract

These days, because of the coronavirus, all countries are introducing online university systems. Online universities have the advantage of allowing students to take classes anytime, anywhere, 24 h a day, but lectures are given in a non-face-to-face manner between instructors and students. Thus, while students are taking classes on a web-based basis, the problem arises that concentration on the lectures is significantly reduced. Therefore, in order to solve these problems, in this paper, we propose a level-wise learning algorithm based on the difficulty level of the test problem, and we present the simulation results. In order to improve this problem, in this paper, we propose an automatic music recommendation algorithm based on fuzzy reasoning that can improve the level of learning and lecture concentration, and we show our results on developing a web-based, smart e-learning software. As a result of computer simulation, it was proved that the learning test method, considering by level the difficulty of the test and the incorrect answer rate, was more effective than the existing test method, judged the student’s grades fairly, and improved the risk of unfairly failing the test by 30%.

1. Introduction

The MATLAB-based fuzzy inference system is based on the fuzzy set mindset, introduced by Professor L. A. Zadeh of the University of Berkeley in 1965, to present ambiguous phenomena, such as natural language, in an open and quantitative manner [1]. In addition, recently, using fuzzy theory, products applied to smart home appliances, unmanned vehicles, and automatic military control have emerged. In our commonly known proposition or set, we only deal with objectively clear-meaning things such as true or false. Unlike such an ideal situation, however, in real life, there are problems where nothing is classified as true or false. Thus, fuzzy theory was born to solve these ambiguous criteria [2].

In this paper, we aim to develop a learning student-level test based on fuzzy rule-based levels on online learning and offline learning to evaluate students’ understanding of lectures in real time. This is because teachers teach the same lecture to students in class, but there are excellent students who understand the lecture and students who do not understand the lecture. In other words, students’ understanding of lectures is not the same even if teachers conduct the same lectures. A smart e-learning course that is held online has the advantage that anyone can easily take it anytime, anywhere. However, when students continue to watch video lectures on their computer monitors for more than 30 min, the fatigue of their eyes increases and their concentration on the lecture decreases a lot. In order to improve this problem, in this paper, we propose algorithms for music therapy and lighting therapy and present the results of the simulation. In addition, when a student lacks understanding and basic knowledge of the lecture after completing an online video course, we provide basic classes before the student takes the next stage of the lecture. After completing the online video course learning, we propose level-specific learning by providing advanced class courses to students who have abundant understanding and basic knowledge of the lecture [3,4,5,6]. In this paper, we propose an algorithm to solve this problem. We propose a learning algorithm that allows instructors to grasp students’ lecture comprehension and adjust the level of lectures in the next class. In order for instructors to improve students’ lecture concentration while giving online lectures, we propose a learning SW development algorithm for each level and complete a simulation. In addition, by using fuzzy rules in this paper, we propose and simulate an algorithm that evaluates learning according to the student’s level, based on the incorrect answer rate.

2. Smart E-Learning Theory

No matter how good a lecturer is, it is not possible to know the degree of students’ understanding of the class and which subjects are weak during the class. In this study, we develop a level-wise learning SW to address these problems. In other words, based on the blended e-learning technique (online learning + offline learning), we allow instructors to evaluate the degree of students’ understanding of lectures in real time during classes [7,8]. This is because, even in the same class, there are excellent students who understand more than 60–70% of the lecture, and there are students who understand only 30–40% of the lecture. In other words, even if the same lecture is conducted, the students’ level of lecture understanding is not the same. In order to solve this problem, in this paper, we developed a complementary algorithm that had the function of giving lectures by level by automatically analyzing students’ weak subjects on the basis of blended learning; using artificial intelligence technology, we tried to conduct a smart e-learning study based on web and automatic judgment of weak subjects [9,10]. In online universities, students take lectures on a web-based basis, so students take an exam and click a button to manage their scores. Using the algorithm we developed, instructors can check the grades of students who took the test a while ago on the screen in real time. Since online courses are taught on a web-based basis, students can select 10 correct answers from questions 1 to 10 on a multiple-choice test answer sheet and click the submit button to calculate the score immediately [11,12,13].

Figure 1 describes the algorithm for the software implementation in order to evaluate learning by level. In other words, students who have obtained the same score of 60 are distinguished according to the difficulty of the test problem. Students who took high-level exam questions can pass the qualification exam to take advanced class lectures, and students who took the low-level exam questions fail the qualification exam to take advanced classes. In other words, suppose that a student gets a score of B after taking an online class and has a 70% understanding of the class. However, assuming that 85% of the students who take the online math test earn a B grade or higher because the online math test is so easy, there arises a problem that we cannot be sure whether the level of lecture comprehension is necessarily 0.7.

CF (H, E) = 0.7,

CF (H2, E) = 0.5,

CF (32, E) = 0.3

The logical product of hypotheses:

MB (H1 ∧ H2, E) = min (MB (H1, E), MB (H2, E))

MD (H1 ∧ H2, E) = max (MD (H1, E), MD (H2, E))

The logical sum of hypotheses:

MB (H1 ∨ H2, E) = max (MB (H1, E), MB (H2, E))

MD (H1 ∨ H2, E) = min (MD (H1, E), MD (H2, E))

According to the existing logic, we only think of CF (H, E) under the assumption that E is certain. However, if there exists also uncertainty in E (i.e., if the certainty with respect to E itself is not 1), then the certainty factor CF (H, E) of the rule IF E THEN B should also be replaced by CF (H, e) as follows: CF (E, e) shows that the certainty factor of evidence E, a condition of the rule, is also expressed as e, the pre-defined evidence, which is otherwise observed. Here, if CF (E, e) = 1, the certainty factor due to the evidence E of H is indicated only by CF (H, E).

Rewriting the above equation,

CF (H, e) = (MB (H, E) − MD (H, E)) × max (0, CF (H, e))

where the use of ‘max’ means that all CF (H, e) < 0 are considered 0. Let us assume that there are several pieces of evidence that make up the condition. If there is a rule where the conditional statement is an AND combination of three evidences, as follows:

IF E1 AND E2 AND E3 THEN H

where, if the following values

CF (H, E) = CF (H, E1 ∧ E2 ∧ E3) = 0.7

MB (E1, e) = 0.5, MD (E1, e) = 0, CF (E1, e) = 0.5

MB (E2, e) = 0.6, MD (E2, e) = 0, CF (E2, e) = 0.6

MB (E3, e) = 0.3, MD (E3, e) = 0, CF (E3, e) = 0.3 are given, then we get the following:

$$\begin{array}{l}\mathrm{C}\mathrm{F} \left(\mathrm{E},\mathrm{e}\right) =\mathrm{C}\mathrm{F} (\mathrm{E}1\wedge \mathrm{E}2\wedge \mathrm{E}3,\mathrm{e})\\ =\mathrm{M}\mathrm{B} (\mathrm{E}1\wedge \mathrm{E}2\wedge \mathrm{E}3,\mathrm{e}) -\mathrm{M}\mathrm{D} (\mathrm{E}1\wedge \mathrm{E}2\wedge \mathrm{E}3,\mathrm{e})\\ =\mathrm{m}\mathrm{i}\mathrm{n} \left(\mathrm{M}\mathrm{B} \left(\mathrm{E}1,\mathrm{e}\right),\mathrm{M}\mathrm{B} \left(\mathrm{E}2,\mathrm{e}\right),\mathrm{M}\mathrm{B} \left(\mathrm{E}3,\mathrm{e}\right)\right)-\\ \mathrm{M}\mathrm{a}\mathrm{x} \left(\mathrm{M}\mathrm{D}\left(\mathrm{E}1,\mathrm{e}\right),\mathrm{M}\mathrm{D} \left(\mathrm{E}2,\mathrm{e}\right),\mathrm{M}\mathrm{D} \left(\mathrm{E}3,\mathrm{e}\right)\right)\\ =\mathrm{m}\mathrm{i}\mathrm{n}\left[0.5,0.6,0.3\right]\\ \mathrm{C}\mathrm{F} \left(\mathrm{H},\mathrm{e}\right)=\mathrm{C}\mathrm{F} \left(\mathrm{E},\mathrm{e}\right) \mathrm{C}\mathrm{F} \left(\mathrm{H},\mathrm{E}\right)\\ =0.3\ast 0.7\\ =0.21\end{array}$$

3. Student Level Test Concept

In this paper, we propose an algorithm that allows students to learn by level according to their acquired test scores [14,15,16]. However, the following problem arises in the existing method of online midterm examination: According to the existing test score evaluation method, scores are calculated and evaluated into four types of A, B, C, and D. In other words, if the online test problem is easily presented, 100% of students can receive A or B credits and on the contrary, if the test problem is difficult, about 80% of students can get C or D credits. In this paper, we propose a level-wise learning algorithm that allows students to take classes that fit their level.

In Figure 2, we explain an algorithm that allows instructors to give lectures to students by levels even if they have the same 60 scores, depending on the difficulty and the error rate of the test. This is because even with the same 60 points, if a score is obtained on a difficult exam, the grades change to a downward trend, so there are few high-ranking students, and the number of low-ranking students rapidly increases. Therefore, even with the same score of 60 points, students with 55 points should be allowed to take intermediate classes if the number of students who get the correct answer is low because the test is difficult. However, when students learn only by level by test scores, problems arise. In this paper, we have proposed an algorithm that divides students into three groups: advanced classes for students with 80–100 points, intermediate classes for students with 60–70 points, and basic classes for students with less than 60 points. We assume that the web-based mathematics lecture consists of steps 01 to 05 until students take the midterm exam and steps 06 to 10 until they take the final exam. In this paper, we propose a level-based learning algorithm to efficiently manage existing lecture-taking systems. In the student evaluation system, in order to continue taking the 02-level lecture, after the students have finished the online lecture in the 01st stage, they must take the online test on a web basis in order to evaluate their understanding of the 01-level lecture. However, problems arise in the method determining that a student with a test score of 60 or higher is PASS and a student with a test score of less than 60 is FAIL. This is because assuming that if 5 sites for online mathematics lectures provided by the web are operated, if the mathematics test questions of a certain site are difficult, the number of FAIL students increases.

On the contrary, if the mathematical problem provided by a site is easy, the number of students who get PASS increases. Therefore, even if the test score is the same 70 points, some students will PASS and some students will FAIL. In this paper, we propose a level-based learning algorithm based on fuzzy rules to solve these problems [17,18,19].

As shown in Figure 3, if one intensively watches a lecture video for more than 30 min, the fatigue of one’s eyes increases, and concentration on the lecture decreases. Therefore, to improve this problem, the results of a questionnaire that proposes a music therapy and treatment algorithm for student personality and lecture concentration are described [20,21]. Simulation results showed that when studying science subjects, blue lighting was effective; when studying memorization subjects, yellow lighting improved the learning effect; and red lighting was effective in arts subjects [22,23,24].

4. Computer Simulation for Learning by Level

Many students think that the assessments of their school are not reasonable because the students often do not solve easy problems. They say that it was only a mistake. However, some students do not solve difficult problems. In order to solve this problem, we propose a fuzzy rule for reasonable assessment. Fuzzy relation means the meaning of “relation” is fuzzy. There are students who can understand 80% of the lecture and also students who understand below 50% of lecture, even if they take the same class.

Figure 4 shows a fuzzy system method that is used to analyze the score of many students. It is an estimation process into five different conditions that are used to predict the ability of each student. With this approach, the assessment prompts the learning effort in e-learning. The items of

Table 1. Input–output data for learning by level and inference for learning by level.

Input Data			Level Test
Test Score	Item Difficulty	Wrong Answer Rate	Conventional Method	Proposed Method
90	Small	Big	Pass	Pass
85	Med	Big	Pass	Pass
60	Small	Big	Pass	Fail
60	Big	Big	Pass	Fail
57	Med	Small	Fail	Pass
75	Big	Big	Pass	Pass
62	Big	Big	Pass	Fail
90	Small	Big	Pass	Pass
58	Big	Big	Fail	Pass

become the inputs of the fuzzy rule. After the process of fuzzy learning, we can get the state of the learners in the cyber university.

In a class, it is not easy to know how many students can understand the lecture. If students who have different abilities take the same class, the lecture will focus on students who understand most parts of it. Accordingly, to overcome this problem, our proposed system provides a proper level of lecture for each student. However, to run level learning, it is hard to expect high effect with the same textbook. The superior class should use a difficult textbook and the inferior class should use an easy textbook to get a good effect.

This is a private-level education system in a traditional education system. In this article, we apply a full duplex learning scheme to an online cyber university because many students do not completely understand their subject. For this reason, they suffer from difficult problems at the next level. Therefore, proper feedback is needed for them to completely understand their subject.

Figure 5 describes the learning flow chart for each level. As explained in Section 2, if the test is over and the test score is 60 or higher, the student passes, and they fail if it is less than 80, the following problems arise: Even if the student test score is the same 60 points, the difficulty of the test is difficult, and the rate of incorrect answers is high or low.

This is because even with the same 60 points, if the test difficulty is difficult, the grades change to a downward trend, so there are few high-ranking students, and low-ranking students rapidly increase. Therefore, in Figure 6, if the test difficulty is difficult even at 60 points, and there are few students who answer the test questions correctly even if the test score is 55 points, the algorithm that can make intermediate class lectures is explained using fuzzy rules.

Figure 1 illustrates the initial menu screen of smart eLearning. According to Figure 3, if a student who takes a lecture at an online university watches a video intensively for more than 30 min through a computer monitor, concentration on the lecture decreases a lot due to increased eye fatigue. In this paper, in order to improve this problem, we propose and simulate an algorithm for music therapy and lighting therapy suitable for students’ personalities and fatigue of lecture concentration through surveys from students taking classes. By the definition of question discrimination, students with a high total score on an examination tend to show a high score on a question. In other words, if the value of the correlation coefficient between the two scores is large, the distinction of the question may be high, and the formula for obtaining the correlation coefficient is as follows:

$$r_{bis}=\frac{M_R-M_W}{S_t}\times \frac{P\left(1-P\right)}{Y}$$

$M_R$: Average score of students who responded to correct answers.

$M_W$: Average score of students who responded to wrong answers.

$S_t$: Standard deviation of the overall score distribution.

$P$: Correct answer rate of all students.

$Y$: Values on the longitudinal axis, dividing $P$ (correct answer part) and 1 −$P$ (the incorrect answer part) in the normal distribution curve.

Figure 7 describes the process in which test questions are automatically presented depending on the level of difficulty. In other words, in this paper, we propose an algorithm and perform simulations in order to automatically present test questions wherein we select the difficulty level of the subject as beginner, intermediate, and advanced, according to the level of the student.

In Figure 8, we perform a simulation that allows students to adjust the passing score for taking upper-level lectures using fuzzy rules. The input membership function consists of test scores, question difficulty, and incorrect answer rate, and the output membership function consists of learning passing scores for each level. If you substitute a value for a fuzzy input variable, the learning output score by level is displayed. The existing smart e-learning method simply calculates the learning score for each level with student test scores. In this paper, as previously described, we propose an algorithm that allows us to select level-specific learning to fit the student level taking into account the question difficulty and the error rate, and we conduct a computer simulation.

IF SCOR E = Medium Level and

Wrong answer rate = High Level and

Absent = Low Level and

Then

LEVEL TEST = CNF 70

where CNF 70 means that the certainty factor of RULE is 70%. If the certainty factor is indicated by the existing method without using the fuzzy rule, the probability of being classified as C credit is always predicted to be 100%. In this paper, we propose an algorithm and conduct computer simulations that can judge the certainty factor of learning by level as 70% by considering the incorrect answer rate and absence rate rather than evaluating learning by level only with students’ grades. If the certainty factor is given as 80, the certainty factor for the conclusion is 0.8 × 0.7 = 0.56.

Studying for a long time does not mean that one can understand the whole content. If one studies most efficiently, one can achieve one’s desired goals in a short amount of time. Figure 9 explains the process of judging the Sasang constitutional test and recommends study methods suitable for students’ personalities in order to improve the grades of the students taking the class [25,26,27,28]. It is good for students to know the characteristics of their Sasang constitution and study according to their constitution.

Table 1 describes the input–output data for learning by level and the inference for learning by level. In the existing method, if the score was 60 or higher, the level-specific learning test was passed. In this paper, however, we propose and simulate a level-based learning algorithm based on fuzzy rules. That is, even if a score below 60 is a Fail, if the question is a test with a high error rate due to difficulty, a Pass score is calculated.

Figure 10 explains the results of the learning simulation by level and the process of automatically recommending textbooks. To finish a test and improve one’s grades, it is important to take lectures that suit one’s level and select a textbook that is appropriate for one’s learning ability. In this paper, in order to solve this problem, we propose a level-specific learning algorithm using fuzzy rules and perform computer simulations.

Figure 11 illustrates the experimental results of an automated recognition of the error rate and a systematic simulation of the lecture. This paper presents and simulate algorithms that evaluate a student’s lecture understanding based on fuzzy rules, considering the difficulty and the wrong answer rate, in order to effectively learn level by level. Previously, even after the exam, a student’s answer sheet grades were treated as passing if they were 60 points or higher, and if they were less than 60 points, they were treated as failing. However, in this paper, in order to tackle these problems, we use fuzzy rules. Development of a lecture comprehension system, an automatic recommendation algorithm for textbooks, and study methods for students with poor test scores are proposed, and computer tests are simulated.

The source of the computer simulation algorithm for automatic judgment of learning by level completed in this paper is shown in Algorithm 1 as follows:

Algorithm 1

[System]         Name = FUZZY-LEVEL-TEST         Type = ’mamdani’         Version = 2.0 NumInputs = 3 NumOutputs = 1 NumRules = 9 AndMethod = ‘min’ OrMethod = ‘max’ ImpMethod = ‘min’ AggMethod = ‘max’ DefuzzMethod = ‘centroid’ [Input1] Name = ‘SCORE’ Range = [0 100] NumMFs = 3 MF1 = ‘BAD’: ‘gaussmf’, [15 0] MF2 = ‘AVERAGE’: ‘gaussmf’, [15 50] MF3 = ‘Exellent’: ‘gaussmf’, [15 100] [Input2] Name = ‘Difficulty’ Range = [0 100] NumMFs = 3 MF1 = ‘SMALL’: ‘trapmf’, [0 0 10 37.17] MF2 = ‘BIG’: ‘trapmf’, [58.33 90 100 100] MF3 = ‘MIDDLE’: ‘trimf’, [20.64 51.14 70.56] [Input3] Name = ‘Wrong-ANSWER’ Range = [0 100] NumMFs = 3 MF1 = ‘small’: ‘trapmf’, [−36 −8 12 36] MF2 = ‘medium’: ‘gaussmf’, [12.74 50] MF3 = ‘big’: ‘trapmf’, [64 85 104 136] [Output1] Name = ‘PASS’ Range = [0 118] NumMFs = 3 MF1 = ‘LOW’: ‘trapmf’, [−143.9 1.298 8.26 54.16] MF2 = ‘Average’: ‘gaussmf’, [10.01 59.15] MF3 = ‘HIGH’: ‘trapmf’, [64.78 109.3 127.5 187.7] [Rules] 3 0 0, 3 (1): 2 2 0 0, 2 (1): 2 3 1 3, 3 (1): 1 3 0 3, 3 (1): 2 2 0 2, 2 (1): 2 3 1 0, 3 (1): 2 1 2 0, 1 (1): 2 0 0 1, 1 (1): 2 0 0 3, 3 (1): 2

In the student evaluation method, if a student’s answer sheet score was 60 or higher, it was treated as pass, and if it was less than 60, it was treated as disqualified. In this paper, we proposed a level-specific learning algorithm that uses fuzzy reasoning rules for test difficulty and student incorrect answer rate, so that even students with 50 points can pass the test if the difficulty of the test question is difficult and the student error rate is high. On the contrary, if the difficulty of the test question is easily answered and the student’s incorrect answer rate is low, the student who received 70 points also failed the test. These results were based on

Table 2. Simulation results of automatic judgment of learning by level based on incorrect answer rate.

NAME	SCORE	DIFFICULTY	WRONG ANSWER	CORRECT ANSWER	SCORE INCREASING RATR	NON FUZZY	FUZZY
ALICE	90	SMALL	BIG	MEDIUM	SMALL	PASS	PASS
TOMAS	60	BIG	SMALL	BIG	SMALL	PASS	FAIL
CINDY	50	SMALL	BIG	SMALL	SMALL	FAIL	PASS
KENT	80	MEDIUM	MEDIUM	SMALL	BIG	PASS	PASS
JOHN	55	MEDIUM	SMALL	BIG	SMALL	PASS	FAIL
LEE	55	SMALL	BIG	SMALL	SMALL	FAIL	PASS
KIM	80	SMALL	BIG	MEDIUM	SMALL	PASS	PASS
HONG	70	VERY BIG	BIG	MEDIUM	BIG	FAIL	PASS
SMITH	70	MEDIUM	MEDIUM	SMALL	MEDIUM	PASS	PASS
MARY	65	SMALL	BIG	SMALL	SMALL	FAIL	PASS
ALICE	60	MEDIUM	BIG	MEDIUM	BIG	PASS	FAIL

. Computer simulation results, test difficulty, wrong answer rate, and the learning test method by level taking into account the rate of increase in grades proves the effect of improving the student’s grades fairly and improving the rate of unfairly failing the test by more than 30%, compared to the existing test method.

Karen Swan et al. encourages online collaborative activities to evaluate joint discussions, small group collaborations, and collaborative exams. In each case, they provide rationale and practical advice for evaluation [29]. S Vonderwell et al. evaluated online learning based on student discussion [30]. In the same manner, LR Kearns evaluated with presentations and online interviews [31]. Next, R Goodfellow, and MR Lea described the fitting of the rhetorical demands imposed in the learning environment, i.e., and increasingly fitting to writing, evaluating the current approach to online learning, which tends to cooperatively towards a constructivist perspective up to the current method [32]. M Graff evaluated participants’ ratings for each online engagement. [33].

As mentioned above, the evaluation criteria for existing studies vary depending on the evaluator’s environment. However, this study evaluates examination, and presents evaluation criteria using fuzzy reasoning rules that consider test difficulty and student incorrect answer rates according to the level of student groups.

5. Conclusions

In this paper, we present a program we have developed to simulate smart e-learning on a web basis. It is predicted that smart e-learning systems will be activated in the 21st century. In addition, based on level-specific learning, we have developed systems for automatic textbook recommendations and automatic warnings for incorrect answer rates.

In particular, in massive open online courses (MOOC) based on online distance lectures, the graduation rate of students is very low, at less than 10%. In this paper, in order to improve these problems, we proposed an automatic warning system for incorrect answer rates based on level-based learning as well as an algorithm for calculating test scores based on difficulty and presented the results of the simulation. In particular, we have proposed an algorithm that accurately calculates test scores for each level in online and offline schools by using fuzzy rules in order to improve the test grade system for each level based on difficulty. In addition, in online classes, when one is taking video lectures for more than an hour, concentration on lectures is greatly reduced. Therefore, in order to improve lecture concentration, we have proposed and simulated art therapy and music therapy research, study patterns considering the Sasang constitution, and automatic recommendation algorithms.