A Critical Review of Artificial Intelligence for Assessment: From Promise to Practice

Abstract

The incorporation of Artificial Intelligence (AI) into evaluation processes in education, recruitment, and research has significantly accelerated, propelled by advancements in large language models (LLMs) and machine learning. This critical review amalgamates evidence from systematic reviews, empirical studies, and ethical frameworks published from 2018 to 2026 to assess AI's role in evaluation. The review looks at the technical state of AI-based assessment, talks about the problems of algorithmic bias and fairness, points out the epistemological limitations of machine scoring, talks about the gaps in regulatory and ethical accountability, and thinks about the paradox of using AI to find AI-generated content. Our analysis finds that while AI can grade consistently and quickly, the evidence for its accuracy is shaky. Most performance tests for AI (benchmarks) are not statistically sound, efforts to reduce bias are fragmented, and current quality standards fail to address AI's unique failures. Bias mitigation efforts are disjointed, and current assessment quality criteria do not adequately address AI's distinct failures.