An Automatic Assessment System for Marking Programming Exercises with Random Output

• Main Authors: Yu-Tzu Wu, 吳育姿
• Other Authors: Shang-Hsien Hsieh
• Format: Others
• Language: zh-TW
• Published: 2017
• Online Access: http://ndltd.ncl.edu.tw/handle/mkf89k

碩士 === 國立臺灣大學 === 土木工程學研究所 === 105 === Implementing automatic assessment tools on laboratory or homework exercises in programming courses is a way to provide timely feedback for students as well as lower the instructors’ marking burden. However, if the exercises involve random outputs, which are not judged against a gold standard of comparison, system-level source code modification is often required on a ubiquitous online judging system. This research develops an online judging system as a platform, aiming at adopting a variety of assessment specifications. This research also introduces a simple procedure for customizing judge scripts as the core of the online judging system and deals with all textual output problems in 12 aspects: two options for determining whether a problem requires input of test data; three options for classifying a problem with or without random outputs; and two options for matching special output symbols. By designing the judge scripts based on theses 12 aspects, this approach was an excellent solution that dealt with almost half (45.95%) of the exercises that other online judging systems could not handle. With the advantage of preserving submitted source codes, the online judging system not only replaces instructors’ grading work but also enables plagiarism detection for each exercise. Thus, the online judging system frees instructors’ from tedious routine tasks, enabling them to offer further assistance to students in need. During the experiment in a programming course, students could successively correct and re-submit their source codes until their codes passed the test, while the instructors could inspect quality of students’ source codes after class for improving pedagogy. To investigate the future potential of the online judging system, this research analyzed correlations between the students’ performance and data collected from the system. It was found that making a large number of repeated compiling errors is one of the main behaviors of the weak students (Max. |r|=0.52; p<0.01). Therefore, this research proposes that the instructor may set a compiling error threshold to identify the weak students at an early stage so that extra tutoring can be arranged.