Nafi'’atun Amaliya, Harjono, Sri Susilogati Sumarti, Ella Kusumastuti, Dimas Gilang Ramadhani
This study develops an innovative assessment system based on Artificial Intelligence (AI) and causal inference to objectively evaluate teachers' explaining skills. Traditional assessments often suffer from subjectivity and inconsistency, limiting their effectiveness in capturing the complex and dynamic nature of teaching skills. To overcome these limitations, this research integrates a black box AI model (Google Gemini 1.5 Flash LLM) and a glass box approach (multivariate linear regression) to provide transparent and interpretable assessments. Using 200 video transcripts from various teaching contexts, the results indicate a high correlation (r = 0.91) between the AI-generated scores and regression model predictions, confirming the validity and reliability of the approach. Causal analysis through Directed Acyclic Graphs (DAG) and Propensity Score Matching (PSM) identifies critical teaching indicators, such as “Adaptation to Student Understanding” (ATE = 0.41) and “Wait Time” (ATE = 0.48), which significantly impact teaching effectiveness. Counterfactual simulations further reveal potential score improvements by up to 0.6 points when enhancing these key areas. The proposed method provides systematic, transparent, and actionable insights, contributing significantly to improving educational quality through precise and evidence-based teacher evaluations. This research also aligns with the Sustainable Development Goals (SDGs), particularly SDG 4 (Quality Education), SDG 9 (Industry, Innovation, and Infrastructure), and SDG 10 (Reduced Inequalities), by promoting equitable, innovative, and high-quality education practices. © 2025 The Authors
Chemistry Education, Faculty of Mathematics and Natural Science, Universitas Negeri Semarang, Indonesia; Chemistry, Faculty of Mathematics and Natural Science, Universitas Negeri Semarang, Indonesia
