Does Student-Submission Allocation Affect Peer Assessment Accuracy?

Hideaki Ohashi, Toshiyuki Shimizu, Masatoshi Yoshikawa

Research output: Contribution to journalArticlepeer-review


Peer assessment in education has pedagogical benefits and is a promising method for grading a large number of submissions. At the same time, student reliability has been regarded as a problem; consequently, various methods of estimating highly reliable grades from scores given by multiple students have been proposed. Under most of the existing methods, a nonadaptive allocation pattern, which performs allocation in advance, is assumed. In this study, we analyze the effect of student-submission allocation on score estimation in peer assessment under a nonadaptive allocation setting. We examine three types of nonadaptive allocation methods, random allocation, circular allocation and group allocation, which are considered the commonly used approaches among the existing nonadaptive peer assessment methods. Through simulation experiments, we show that circular allocation and group allocation tend to yield lower accuracy than random allocation. Then, we utilize this result to improve the existing adaptive allocation method, which performs allocation and assessment in parallel and tends to make similar allocation result to circular allocation. We propose the method to replace part of the allocation with random allocation, and show that the method is effective through experiments.

Original languageEnglish
Pages (from-to)888-897
Number of pages10
JournalIEICE Transactions on Information and Systems
Issue number5
Publication statusPublished - 2022
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence


Dive into the research topics of 'Does Student-Submission Allocation Affect Peer Assessment Accuracy?'. Together they form a unique fingerprint.

Cite this