Sari Dewi Noviyanti, Rudi Hartono, Hendi Pratama, Seful Bahri
—This research tested whether prompt engineering could act as a type of pedagogical mediation to enhance the quality of AI-generated student feedback on EFL students' argumentative essays in an Islamic education system. An initial pool of eight expert raters (four primary raters and four inter-raters) scored the AI-generated feedback from Claude Sonnet 4 using 12 systematically developed prompts to elicit feedback. Raters were asked to score feedback produced by these prompts across four areas of evaluation: pedagogy, linguistics, Islamic content, and AI reliability. The highest rated configuration was Prompt 4 (Feedback-only sequencing, English Lecturer Persona), with a mean rating of 31.00 (out of 35) in all categories. A Friedman test showed there were statistically significant differences among the four evaluative categories, χ2(3) = 30.077, p <.001. Additionally, inter-rater reliabilities were high for each of the possible pairs of raters (r =.89 -.96). Overall, this research suggests that prompt engineering is a potentially viable method of pedagogical mediation, allowing educators to develop more culturally responsive and pedagogically relevant AI-generated feedback systems for Islamic EFL higher education settings. © (2026), (Science and Information Organization). All rights reserved.
English Language Education Department, Semarang State University, Central Java, Semarang, 50229, Indonesia
