Assessing the Critical Failure Factors of AI Chatbots for Research Using ISM Approach

Overview

This study investigates the critical failure factors constraining the effective integration of artificial intelligence chatbots into academic research processes. While prior literature has identified discrete failure factors, the structural interdependencies among these factors remain underexplored. The research applies interpretive structural modeling to analyze hierarchical relationships and causal pathways among identified failure factors within an empirical university context.

Methods and approach

The study employs interpretive structural modeling as the primary analytical technique to map structural relationships among critical failure factors of AI chatbots in research contexts. Data collection and analysis were grounded in a case study of an actual university setting in the Philippines, enabling examination of contextual manifestations and interrelationships of identified failure factors within an institutional research environment.

Key Findings

Analysis reveals a hierarchical structure of failure factors with asymmetric causal influences. Insufficient researcher knowledge and domain expertise emerged as the most influential upstream factor within the structural model, demonstrating primary causal precedence over other failure factors. The findings establish a dependency hierarchy wherein foundational deficiencies in domain-specific knowledge propagate downstream effects across multiple failure categories.

Implications

The results indicate that AI chatbots cannot function as independent research knowledge generators but must remain positioned as auxiliary instruments within research workflows. Authors require foundational domain expertise and first-hand knowledge of their research field to maintain epistemic authority and generate novel contributions. The structural dominance of knowledge deficiency suggests that institutional integration strategies should prioritize researcher capability development and maintain human oversight rather than pursuing unconstrained chatbot autonomy in research processes.