AI meets Mathematics Education: Supporting Instructors in Large Mathematics Classes with Context-Aware AI

Key findings from this study

This research indicates that:

• A fine-tuned lightweight language model achieved 75.3% accuracy in answering course-specific mathematics questions, with instructor evaluation rating 36% of responses as equivalent to or superior to instructor-authored answers.
• Students perceived primary value in response alignment with course materials and immediate availability, yet required instructor verification to establish trust in system outputs.
• Hybrid workflows pairing AI-generated responses with mandatory instructor review provide a feasible mechanism for scaling academic support in large-enrollment courses while preserving pedagogical oversight.

Overview

This case study examines the deployment of a fine-tuned language model to provide automated question-answering support in a large-enrollment Calculus I course. The system operated within a hybrid workflow pairing AI-generated responses with instructor oversight, addressing scalability constraints in providing timely academic support to high-enrollment courses.

Methods and approach

Researchers developed a lightweight language model fine-tuned on 2,588 historical student-instructor interactions from course discussion forums. The model underwent evaluation on 150 representative annotated questions, with benchmark performance assessed by five independent instructors. Post-deployment data collection included student survey responses (N = 105) addressing perceived system utility and trust mechanisms.

Results

The fine-tuned model achieved 75.3% accuracy on the benchmark dataset. Instructor evaluation determined that model-generated responses matched or exceeded instructor answers in 36% of cases. Survey respondents identified two primary system strengths: alignment of responses with course materials and near-instantaneous availability. Students indicated that instructor verification remained essential for establishing confidence in system outputs, suggesting limited autonomous reliability in the student perspective despite quantitative accuracy metrics.

Implications

The hybrid human-AI workflow demonstrates feasibility for scaling instructional support in large courses without displacing instructor authority. The 36% equivalence or superiority rate, combined with student reliance on instructor validation, indicates that AI assistance functions optimally as a supplementary resource rather than a replacement for direct instruction. Integration of fine-tuned models with lightweight architectures may reduce computational overhead while maintaining pedagogical alignment in institutional contexts.

Context-aware model development that leverages institution-specific historical interactions appears necessary for achieving pedagogical relevance. Students' reported valuation of material alignment suggests that generic generative systems may underperform without discipline-specific training data. Future implementations should prioritize transparent hybrid workflows that explicitly embed instructor oversight as a structural component rather than an afterthought.

Scope and limitations

This summary is based on the study abstract and available metadata. It does not include a full analysis of the complete paper, supplementary materials, or underlying datasets unless explicitly stated. Findings should be interpreted in the context of the original publication.