UAJY Handbook RAG Chatbot Separates FAISS Search From Gemini Embeddings

Modular RAG Architecture for Academic Handbooks

The UAJY Academic Document RAG Chatbot, available on GitHub, presents a robust, inspectable implementation for developers building Retrieval Augmented Generation (RAG) systems. This open-source project, specifically tailored for the 2025/2026 academic handbook of Universitas Atma Jaya Yogyakarta’s Faculty of Industrial Technology, showcases a modular approach to RAG. It effectively separates concerns, particularly the generation of embeddings from the vector storage and search mechanism. This architectural choice offers greater flexibility and control, allowing developers to swap components without disrupting the entire pipeline. The project packages a Streamlit-based assistant, providing a user-friendly interface for interacting with the handbook’s content. Beyond the core RAG functionality, it includes features for document ingestion, local vector search, cited answer generation, refusal controls, and a 20-question evaluation harness, all within a single, coherent codebase.

The ingestion pipeline is designed for offline processing, meaning it can be run entirely without an internet connection once the initial setup is complete. It leverages the `pdfplumber` library to extract both text and tabular data from the PDF handbook. Following extraction, the content is segmented into semantically coherent chunks, or paragraph-based segments. This chunking strategy is crucial for effective retrieval, ensuring that the context provided to the language model is relevant and focused. Each chunk then has a 3,072-dimensional embedding vector generated using Google's `gemini-embedding-001` model. These vectors are not directly stored in a cloud-based vector database; instead, they are persisted locally within a FAISS (Facebook AI Similarity Search) index. This local indexing significantly speeds up search operations and ensures data privacy, as sensitive academic information remains on the user’s machine. Crucially, the FAISS index is augmented with metadata, including page numbers and relevant headings from the original document. This metadata is vital for providing precise citations and context in the generated answers. The repository indicates that the resulting index contains approximately 350 distinct chunks, each represented by a vector and associated metadata, enabling efficient similarity searches for user queries.

Decoupling Embedding Generation and Vector Search

A key design principle demonstrated by the UAJY chatbot is the explicit separation of the embedding generation process from the FAISS vector index. While `gemini-embedding-001` is used to create the high-dimensional vector representations of the document chunks, these embeddings are then stored and managed by FAISS. This is a deliberate architectural choice that offers several advantages. Firstly, it allows for easier experimentation with different embedding models. If a new, more performant or specialized embedding model becomes available, developers can integrate it into the pipeline without needing to rebuild the entire FAISS index from scratch, provided the output dimensionality is compatible or can be handled by FAISS. Secondly, FAISS is chosen for its efficiency in similarity search, particularly for large datasets, and its ability to run entirely offline. By decoupling, the system doesn’t rely on a specific embedding API for real-time retrieval queries; the embeddings are pre-computed and stored. This approach is particularly beneficial for applications where latency is critical or where internet connectivity is unreliable. The pre-computation of embeddings ensures that when a user submits a query, the system can immediately perform a similarity search against the FAISS index to find the most relevant document chunks. The retrieved chunks, along with their associated metadata (page number, heading), are then passed to a language model (implicitly Gemini, given the embedding model used) to synthesize a coherent, cited answer. This separation creates a flexible RAG architecture that is both performant and adaptable to evolving NLP technologies.

Implementation Details and Evaluation

The project’s implementation details are thoroughly documented within the repository, making it an excellent resource for developers looking to replicate or adapt this RAG pattern. The pipeline begins with the PDF ingestion, where `pdfplumber` handles the complex task of parsing PDF structures, including text and tables. The subsequent chunking strategy balances semantic meaning with manageable segment sizes, a critical factor in retrieval accuracy. For instance, if a chunk is too short, it might lack sufficient context; if it's too long, it might contain too much irrelevant information, diluting the signal when passed to the LLM. The use of 3,072-dimensional embeddings from `gemini-embedding-001` suggests a focus on capturing rich semantic nuances within the document content. FAISS, as the local vector database, provides near real-time search capabilities. Its efficiency stems from advanced indexing techniques that allow for rapid approximate nearest neighbor searches, even with millions of vectors. The inclusion of a 20-question evaluation harness is a standout feature for assessing the chatbot’s performance. This harness likely involves a predefined set of questions designed to test various aspects of the chatbot’s knowledge retrieval and answer generation capabilities. By evaluating against these questions, developers can quantitatively measure the effectiveness of their RAG pipeline, identify areas for improvement, and benchmark different configurations. The refusal controls are also a critical component, ensuring the chatbot only answers questions based on the provided handbook content and avoids hallucination or responding to out-of-scope queries. This is achieved by checking the relevance of retrieved documents against the user's query and potentially using a secondary classifier or confidence score before invoking the LLM.

Broader Implications for RAG Development

The UAJY Academic Document RAG Chatbot’s architecture offers valuable insights for the broader RAG development community. By demonstrating a clean separation between embedding generation and vector storage/search, it provides a blueprint for building more adaptable and maintainable RAG systems. This modularity is essential in the rapidly evolving field of AI, where new models and techniques emerge frequently. Developers can now more confidently experiment with different components, knowing that the core logic is well-defined and easily extensible. The emphasis on offline processing and local FAISS indexing also highlights a growing trend towards privacy-preserving and self-contained AI applications, particularly for sensitive or proprietary data. This approach reduces reliance on external APIs, lowers operational costs, and enhances data security. The inclusion of a comprehensive evaluation harness is another significant contribution, providing a standardized method for assessing RAG performance. This will be invaluable for researchers and practitioners aiming to benchmark their own RAG implementations against a clear set of criteria. The project serves as a practical, hands-on example that demystifies the complexities of building custom RAG solutions, empowering developers to create more sophisticated and reliable AI-powered applications from their own document repositories.