A systematic review of the impact of GenAI on learning performance, AI hallucinations, and problem-solving in computer science education

Key findings from this study

• The review identifies that GenAI produces dual impacts in computer science education: negative effects from hallucinations and over-reliance degrade learning performance, while positive outcomes emerge when GenAI is embedded within pedagogically grounded and equitable structures.
• The authors report that hallucinated outputs increase extraneous cognitive load during programming and debugging tasks, disrupting error detection and self-monitoring mechanisms essential for learning.
• The review demonstrates that structured GenAI environments promoting reflective practice, solution verification, and strategic adjustment enhance problem-solving skills and personalized learning outcomes.

Overview

This systematic review synthesizes 64 empirical studies examining how generative AI shapes learning outcomes, hallucination effects, and problem-solving capacities in computer science education. The analysis integrates constructivist, sociocultural, cognitive load, adaptive learning, and metacognitive frameworks to assess GenAI's impact across programming, debugging, algorithmic reasoning, and computational problem-solving contexts. The review reveals a dual pattern: GenAI produces adverse effects when outputs contain hallucinations or when deployment lacks pedagogical grounding, but yields positive learning gains when embedded within structured, equitable educational environments.

Methods and approach

The authors conducted a systematic review across 64 empirical studies in computer science education. The analysis employed five theoretical lenses: Constructivist, Sociocultural, Cognitive Load, Adaptive Learning, and Metacognitive Learning theories. The integrative framework examined how GenAI-driven adaptivity, output quality, hallucination dynamics, and cognitive-affective regulation influence learner interpretation, cognitive processing, and outcomes.

Results

GenAI demonstrates polarized effects contingent on implementation context. Negative impacts include increased extraneous cognitive load from hallucinated outputs, heightened over-reliance on system-generated solutions, and disrupted error detection and self-monitoring mechanisms. These effects correlate with impaired learning performance and widened educational disparities, particularly in low-resource settings and among culturally and linguistically diverse learners.

Positive learning outcomes emerge when GenAI operates within pedagogically grounded structures that promote reflective practice. Under such conditions, GenAI scaffolds self-monitoring, solution verification, and strategic adjustment processes. This support strengthens problem-solving skills, sustains engagement, and enables personalization aligned with individual learner needs.

The review establishes hallucination dynamics, learning performance, and problem-solving as interconnected dimensions requiring simultaneous consideration in GenAI-supported computing education design.

Implications

Institutional design of GenAI-supported learning environments must prioritize pedagogical grounding and equity considerations to harness positive effects while mitigating hallucination-induced cognitive load. Educational institutions deploying GenAI in programming contexts should implement structured guidance emphasizing verification practices, error detection strategies, and metacognitive regulation. Such approaches counter over-reliance patterns and strengthen independent problem-solving capacities.

Resource-constrained institutions require targeted interventions addressing access disparities and culturally responsive support mechanisms for linguistically diverse learners. Curriculum developers should design environments that leverage GenAI's adaptive capabilities while maintaining cognitive balance through explicit verification protocols and self-monitoring scaffolds. The integrative framework positions hallucination management and learning outcome optimization as interdependent design challenges rather than isolated technical issues.

Future research should examine long-term effects of structured GenAI integration on metacognitive development and transfer of problem-solving skills beyond immediate programming contexts.

Scope and limitations

This summary is based on the study abstract and available metadata. It does not include a full analysis of the complete paper, supplementary materials, or underlying datasets unless explicitly stated. Findings should be interpreted in the context of the original publication.