Making School-Set Marks Comparable: Test Difficulty and Hidden Learning Inequality

Job market paper by Peter Courtney (October 2026): a common scale for school-set marks, recovered from students who move from primary to secondary school in South Africa.

Job market paper, October 2026.

Paper (PDF)  ·  Online appendix (PDF)  ·  Online appendix (HTML)

Abstract

Assessment has been found to raise learning system-wide where its results can be compared across schools, yet at most a quarter of countries test every primary student comparably. I recover a common scale of learning outcomes—without a new test—from marks that are incomparable because assessments are more difficult at some schools than at others. Each secondary school sets the same assessments for students from many primary schools within months of arrival, allowing comparison of those primary schools’ marks. Primary schools feed many secondary schools, enabling a two-way fixed-effects estimator to chain this network of comparisons across 1.2 million South African students into a common scale. The gap between rich and poor schools halved on raw marks between 2018 and 2022 but did not narrow on the common scale. Compared with raw marks, about three-quarters of primary schools move to a different quintile of learning. Five years after primary school, the common scale predicts the national school-leaving examination better than raw marks. Easier assessments do not buy a higher-achieving secondary school: at the same raw mark, students whose primary school set easier assessments join lower-achieving classmates. The common scale supplies comparable intake marks, a necessary condition for any value-added measure.

Online appendix

Each link opens the HTML appendix at that appendix.

• Appendix A: Attrition Robustness
• Appendix B: Measurement Error Analysis
• Appendix C: Subject-Specific vs Pooled Estimation
• Appendix D: Province-Specific Results
• Appendix E: Replication
• Appendix F: Sorting Permutation Test Methods
• Appendix G: Cross-Cohort Stability of Calibrated Learning Levels
• Appendix H: Additional Validation Diagnostics
• Appendix I: Comparison with Liu, Gershenson, and Anthenelli (2025)
• Appendix J: Using These Estimates Downstream
• Appendix K: External Validation Against NSC Mathematics Results
• Appendix L: Monte Carlo bounding sweeps
• Appendix M: Revealed-preference test, extended results
• Appendix N: Decomposing \(\hat{\beta}_1 = \lambda\rho\): persistence, reliability and horizon tests
• Appendix O: Deriving the Estimating Equation from First Principles
• Appendix P: Wedge-Supply Exhibits
• Appendix Q: Comparability frameworks
• Appendix R: Checks on the exit-school effects
• Appendix S: AKM diagnostics