Home » Inteligencia artificial » When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation

When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation

When is LLM-based reasoning retrieval worth the cost? Lightweight cost-aware query routing for retrieval-augmented generation
Sergio Arjona-Giner, Faraón Llorens-Largo, Alberto Real-Fernández y Rafael Molina-Carmona (Universidad de Alicante)
Information Processing & Management, Volume 64, Issue 2, Part C, February 2027

https://www.sciencedirect.com/science/article/pii/S0306457326006771

Retrieval-Augmented Generation (RAG) systems face a cost–effectiveness trade-off: inexpensive retrievers are efficient, whereas reasoning-based retrievers can improve retrieval quality at substantially higher computational cost. We introduce CARE (Cost-aware Adaptive REtrieval), a lightweight cost-aware binary routing method that decides whether each query should be handled by an inexpensive Hybrid retriever or escalated to the ReSP (Retrieve, Summarize, Plan) reasoning-based retriever. The routing decision is made by a lightweight logistic-regression gate using query-level and inexpensive probe-retrieval features. The evaluation covers two BEIR question-answering (QA) datasets, HotpotQA and Natural Questions (NQ), comprising 7,914,797 documents and 10,857 benchmark queries in total, of which 4097 constitute the fully held-out test sets used for evaluation. Using dataset-specific thresholds selected independently on the corresponding validation splits (τ=0.30 for HotpotQA and τ=0.60 for NQ), CARE achieves utilities of 0.8390 and 0.7596, respectively. These dataset-specific operating points yield a macro-average utility of 0.7993 and a macro-average expensive-branch activation rate of 49.31%. On HotpotQA, CARE approximately matches ReSP in retrieval effectiveness, although its Recall@100 is slightly lower (0.8770 versus 0.8795). Its 0.0007 utility advantage is therefore driven by the routing-cost term rather than by superior retrieval effectiveness, with expensive-branch activation reduced from 100% to 88.79%. On NQ, CARE provides a clearer cost-sensitive benefit, achieving 0.7596 utility with only 9.82% activation.

Keywords
Retrieval-augmented generation, Adaptive retrieval, Query routing, Cost-aware retrieval, Retrieval efficiency


Leave a comment

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *