Abstract
Generative artificial intelligence (AI) and natural language processing (NLP) applications, particularly those utilizing large language models (LLMs) with Retrieval-Augmented Generation (RAG), fine-tuning and their variations, are receiving widespread attention. However, despite their promise, the dependability of LLM applications remains uncertain. This study introduces a new dimension to improving LLM performance through systematic query classification. A taxonomy-based approach was developed to categorize queries by their likelihood of eliciting accurate responses, from those likely to succeed to those prone to failure. Bloom’s Taxonomy (BT) provided the conceptual direction for assessing and classifying queries using a curated dataset of question-answer pairs mapped to BT levels to evaluate RAG-LLM performance across categories. Building on insights from BT-based analysis, an innovative ‘Taxonomical Query Classifier’ (TQC) framework for LLMs is introduced: Finder (F), Evaluator (Ev), Explainer (Ex), and Generator (G) - collectively termed FEEG. FEEG-TQC classifies queries to improve predictability, accuracy, and enhance human-AI interaction. It differentiates queries based on intent, whether they (F) seek factual information, (Ev) evaluate content, (Ex) explain concepts, or (G) generate new content. By developing the concept of query category variations, this research provides a systematic approach to managing query quality and predictive estimations of LLM response accuracy. The findings contribute to the development of more effective human-AI interaction opportunities for LLM-based generative AI systems, emphasizing query classification as an important factor in improving generative AI systems.