Budi Prasetiyo, Hadid Ramadhan, M. Faris Al Hakim
Short Message Service (SMS) spam remains a persistent security threat due to its high delivery reliability, low operational cost, and widespread use as a communication medium. Most existing SMS spam detection systems rely on supervised machine learning or deep learning models that require large labeled datasets and extensive training, which limits their adaptability to rapidly evolving spam patterns. This study investigates the feasibility of applying zero-shot prompting with large language models (LLMs) for SMS spam detection without task-specific training. The classification task is reformulated as an instruction-guided text generation problem, enabling LLMs to perform spam detection through natural language prompts. A comparative evaluation is conducted across four decoder-only LLMs such as Mistral Small 24B, Gemma 2 9B, Qwen 2.5 7B, and LLaMA 3.1 8B using a unified prompt and deterministic inference configuration. Experiments on a balanced subset of the SMS Spam Collection Dataset demonstrate that all evaluated models achieve competitive spam detection performance, with Mistral Small 24B obtaining the highest accuracy of 95%. Further analysis reveals differences in error distribution, model confidence, and inference efficiency across models, highlighting a practical trade-off between classification robustness and computational cost. These findings indicate that zero-shot LLM-based spam detection provides a flexible and scalable alternative to conventional supervised approaches, particularly in dynamic or data-scarce environments. © 2026 IEEE.
Universitas Negeri Semarang, Department Of Computer Science, Semarang, Indonesia
