Semantic Specialization of Legal LLMs via Large-Scale Fine-Tuning on Jurisdictional MCQs

This paper examines how instruction-based fine-tuning enhances the semantic competence of legal Large Language Models (LLMs) in answering multiple-choice questions (MCQs) drawn from jurisdiction-specific legal examinations. Using Breeze-7B, a Mandarin LLM adapted from Mistral-7B, we fine-tune the model on two corpora containing 5,000 and 70,000 legal MCQs to evaluate how training volume affects accuracy, norm identification, and alignment with local legal reasoning. We evaluate the resulting models across three datasets—the Taiwanese Bar Examination (TBE), the Taiwan Multimodal Legal Understanding benchmark (TMMLU), and MMLU—using both probability-based and zero-shot prompt-based protocols. Our findings show that large-scale fine-tuning substantially improves the model’s ability to recognize legal obligations, interpret statutory structures, and reproduce jurisdiction-specific doctrinal reasoning. Comparisons with general-purpose models such as GPT-3.5 and GPT-4o highlight the trade-offs between broad linguistic generality and jurisdictional semantic specialization. We argue that MCQs function as structured, scalable signals for assessing and cultivating legal semantic competence in LLMs. Our results demonstrate that task-specific instruction tuning, combined with jurisdiction-grounded datasets, offers a cost-efficient pathway for developing explainable, localized legal AI systems. This work contributes to ongoing efforts in legal informatics to advance semantic analysis, model adaptation, and digital sovereignty within diverse legal environments.