Online Formative Learning Assessment in Higher Education : Integrating New Scoring Methods with Four-Multiple Choice Assignments
Title
高等教育におけるオンライン形成的学習評価 : 4選択肢課題による新しい採点方法の統合
Online Formative Learning Assessment in Higher Education : Integrating New Scoring Methods with Four-Multiple Choice Assignments
Degree
博士(学術)
Dissertation Number
創科博甲第132号
(2024-03-18)
Degree Grantors
Yamaguchi University
[kakenhi]15501
grid.268397.1
Abstract
The COVID-19 pandemic has significantly transformed higher education, shifting it from traditional classrooms to online platforms. This change requires reassessment and adaptation of educational methods, particularly student assessment. Online formative assessments have become essential for improving teaching and learning outcomes because they provide immediate feedback, enable interactive support, and encourage selfassessment, thereby playing a key role in the learning process.
The multiple-choice test is widely used to assess students. However, the inherent nature of multiple-choice questions poses the risk of obtaining correct answers, even without a genuine understanding of the content. To mitigate this issue, typical measures involve increasing the number of questions. To address this concern, this study implemented a new constraint aimed at enhancing the inherent characteristics of the multiple-choice format. This research objective focuses on investigating innovative scoring methods for formative assessments in online courses that can improve learning in higher education within the context of Yamaguchi University.
This study evaluated the effectiveness of this learning assessment method by employing multiple-choice questions, presenting a practical and efficient approach for online formative learning assessment designed to assess a large student cohort. The new scoring method in this study extends Ikebururo's concepts that introduce partial scoring systems in MCQ design, driving the creation of a new scoring system centered on the "degree of matching. This approach involved comparing the alignment between student responses and the instructor's design, resulting in a detailed five-level scoring system for four-choice questions. This scoring method hinges on evaluating how closely students answers align with the instructor's intended choices. Each question, with its four choices, is akin to a binary process, represented by a 4-digit binary number. Each digit in this comparison corresponds to a specific choice, allowing for a granular assessment of the match between student selection and the ideal answer. This innovative approach steps away from the conventional pass-fail binary system, offering a spectrum of evaluation outcomes. It provides a better understanding of students comprehension by gauging the extent of the alignment between their choices and the instructor's design.
This method can enhance assessment accuracy by capturing the subtleties of student responses beyond mere correctness, earning partial points for partial knowledge or progress via multistep reasoning, promoting critical thinking, recognizing the importance of incremental progress, and capturing the depth of a respondent's knowledge.
Initially, an extensive literature review established a theoretical framework, identifying gaps in the current understanding of online formative assessments. Subsequently, the study examined data collected from graduate students in the 'Advanced Research and Development Strategies' course at Yamaguchi University. The data span two academic years, 2019 and 2020, and provide a comparative view of face-to-face and online Lecturer Formats.
Furthermore, the k-means clustering algorithm was used to analyze student performance using formative assessment scores. This method categorizes student performance into distinct clusters, revealing insights into individual learning behaviors. The k-means method, a popular technique in data mining and pattern recognition, efficiently groups data into 'k' clusters. It is effective for large datasets and versatile across various data types. The technique involves steps such as initialization, assignment, centroid updating, and convergence checking, and is instrumental in identifying performance patterns, enabling the development of more focused educational strategies.
The results demonstrate the potential of the four-choice multiple-choice scoring method to revitalize online formative assessments. The key contributions of this study are as follows:
・Innovative Scoring Method: This study shows how the four-choice method can lead to more dynamic and engaging online assessments. This approach captures student performance more accurately and encourages deeper engagement with the material.
・Enhanced Student Engagement and Understanding: The new four-multiple-choice scoring method significantly affected student engagement and understanding. This fosters an environment in which students are more actively involved in their learning processes, contributing to better comprehension and retention of material.
・Practical Implications for Educators and Institutions: The need to adapt assessment strategies for digital learning, focusing on continuous feedback and personalized learning.
・Educational Technology Contribution: Key insights into adapting assessment strategies for digital learning, emphasizing continuous feedback, and personalized learning.
This dissertation presents a comprehensive examination of new assessment techniques in the context of online learning. This provides a critical roadmap for educators and institutions to adapt to the digital educational environment for more effective and engaging assessment practices in online higher education.
The multiple-choice test is widely used to assess students. However, the inherent nature of multiple-choice questions poses the risk of obtaining correct answers, even without a genuine understanding of the content. To mitigate this issue, typical measures involve increasing the number of questions. To address this concern, this study implemented a new constraint aimed at enhancing the inherent characteristics of the multiple-choice format. This research objective focuses on investigating innovative scoring methods for formative assessments in online courses that can improve learning in higher education within the context of Yamaguchi University.
This study evaluated the effectiveness of this learning assessment method by employing multiple-choice questions, presenting a practical and efficient approach for online formative learning assessment designed to assess a large student cohort. The new scoring method in this study extends Ikebururo's concepts that introduce partial scoring systems in MCQ design, driving the creation of a new scoring system centered on the "degree of matching. This approach involved comparing the alignment between student responses and the instructor's design, resulting in a detailed five-level scoring system for four-choice questions. This scoring method hinges on evaluating how closely students answers align with the instructor's intended choices. Each question, with its four choices, is akin to a binary process, represented by a 4-digit binary number. Each digit in this comparison corresponds to a specific choice, allowing for a granular assessment of the match between student selection and the ideal answer. This innovative approach steps away from the conventional pass-fail binary system, offering a spectrum of evaluation outcomes. It provides a better understanding of students comprehension by gauging the extent of the alignment between their choices and the instructor's design.
This method can enhance assessment accuracy by capturing the subtleties of student responses beyond mere correctness, earning partial points for partial knowledge or progress via multistep reasoning, promoting critical thinking, recognizing the importance of incremental progress, and capturing the depth of a respondent's knowledge.
Initially, an extensive literature review established a theoretical framework, identifying gaps in the current understanding of online formative assessments. Subsequently, the study examined data collected from graduate students in the 'Advanced Research and Development Strategies' course at Yamaguchi University. The data span two academic years, 2019 and 2020, and provide a comparative view of face-to-face and online Lecturer Formats.
Furthermore, the k-means clustering algorithm was used to analyze student performance using formative assessment scores. This method categorizes student performance into distinct clusters, revealing insights into individual learning behaviors. The k-means method, a popular technique in data mining and pattern recognition, efficiently groups data into 'k' clusters. It is effective for large datasets and versatile across various data types. The technique involves steps such as initialization, assignment, centroid updating, and convergence checking, and is instrumental in identifying performance patterns, enabling the development of more focused educational strategies.
The results demonstrate the potential of the four-choice multiple-choice scoring method to revitalize online formative assessments. The key contributions of this study are as follows:
・Innovative Scoring Method: This study shows how the four-choice method can lead to more dynamic and engaging online assessments. This approach captures student performance more accurately and encourages deeper engagement with the material.
・Enhanced Student Engagement and Understanding: The new four-multiple-choice scoring method significantly affected student engagement and understanding. This fosters an environment in which students are more actively involved in their learning processes, contributing to better comprehension and retention of material.
・Practical Implications for Educators and Institutions: The need to adapt assessment strategies for digital learning, focusing on continuous feedback and personalized learning.
・Educational Technology Contribution: Key insights into adapting assessment strategies for digital learning, emphasizing continuous feedback, and personalized learning.
This dissertation presents a comprehensive examination of new assessment techniques in the context of online learning. This provides a critical roadmap for educators and institutions to adapt to the digital educational environment for more effective and engaging assessment practices in online higher education.
Creators
SONEPHACHANH MALAYPHONE
Languages
jpn
Resource Type
doctoral thesis
File Version
Version of Record
Access Rights
open access