AI-Trend-Scout/docs/ADR_001_Hybrid_Search_Architecture.md

ADR 001: Architecture Design for Enhanced Semantic & Hybrid Search

1. Context and Problem Statement

The "Trend-Scout AI" bot currently utilizes a basic synchronous implementation of ChromaDB to fulfill both categorical retrieval (/latest) and free-text queries (/search). Two major issues have severely impacted the user experience:

1. Incorrect Categories in /latest: The system performs a dense vector search using the requested category name (e.g., "AI") rather than a deterministic exact match. This returns semantically related news regardless of their actual assigned category, yielding false positives.
2. Poor Semantic Matches in /search:
   • The default English-centric embedding model (e.g., all-MiniLM-L6-v2) handles Russian summaries and specialized technical acronyms poorly.
   • Pure vector search ignores exact keyword matches, leading to frustrated user expectations when searching for specific entities (e.g., "OpenAI o1" or specific version numbers).
3. Blocking I/O operations: The ChromaStore executes blocking synchronous operations within async def wrappers, potentially starving the asyncio event loop and violating asynchronous data flow requirements.

2. Decision Drivers

• Accuracy & Relevance: Strict categorization and high recall for exact keywords + conceptual similarity.
• Multilingual Support: Strong performance on both English source texts and Russian summaries.
• Performance & Concurrency: Fully non-blocking (async) operations.
• Adherence to SOLID: Maintain strict interface boundaries, dependency inversion, and existing Domain Transfer Objects (DTOs).
• Alignment with Agent Architecture: Ensure the Vector Storage Agent focuses strictly on storage/retrieval coordination without leaking AI processing duties.

3. Proposed Architecture

3.1. Asynchronous Data Flow (I/O)

• Decision: Migrate the local ChromaDB calls to run in a thread pool executor. Alternatively, if ChromaDB is hosted as a standalone server, utilize chromadb.AsyncHttpClient.
• Implementation: Encapsulate blocking calls like self.collection.upsert() and self.collection.query() inside asyncio.to_thread() to prevent blocking the Telegram bot's main event loop.

3.2. Interface Segregation (ISP) for Storage

The current IVectorStore interface conflates generic vector searching, exact categorical retrieval, and database administration.

• Action: Segregate the interfaces to adhere to ISP.
• Refactored Interfaces:

  class IStoreCommand(ABC):
      @abstractmethod
      async def store(self, item: EnrichedNewsItemDTO) -> None: ...
  
  class IStoreQuery(ABC):
      @abstractmethod
      async def search_hybrid(self, query: str, limit: int = 5) -> List[EnrichedNewsItemDTO]: ...
  
      @abstractmethod
      async def get_latest_by_category(self, category: Optional[str], limit: int = 10) -> List[EnrichedNewsItemDTO]: ...
  
      @abstractmethod
      async def get_top_ranked(self, limit: int = 10) -> List[EnrichedNewsItemDTO]: ...
  

3.3. Strict Metadata Filtering for /latest

• Mechanism: The /latest command must completely bypass vector similarity search. Instead, it will use ChromaDB's .get() method coupled with a strict where metadata filter: where={"category": {"$eq": category}}.
• Sorting Architecture: Because ChromaDB does not natively support sorting results by a metadata field (like timestamp), the get_latest_by_category method will over-fetch (e.g., fetch up to 100 recent items using the metadata filter) and perform a fast, deterministic in-memory sort by timestamp descending before slicing to the requested limit.

3.4. Hybrid Search Architecture (Keyword + Vector)

• Mechanism: Implement a Hybrid Search Strategy utilizing Reciprocal Rank Fusion (RRF).
• Sparse Retrieval (Keyword): Integrate a lightweight keyword index alongside ChromaDB. Given the bot's scale, SQLite FTS5 (Full-Text Search) is the optimal choice. It provides persistent, fast token matching without the overhead of Elasticsearch.
• Dense Retrieval (Vector): ChromaDB semantic search.
• Fusion Strategy:
  1. The new HybridSearchStrategy issues queries to both the SQLite FTS index and ChromaDB concurrently using asyncio.gather.
  2. The results are normalized using the RRF formula: Score = 1 / (k + rank_sparse) + 1 / (k + rank_dense) (where k is typically 60).
  3. The combined list of DTOs is sorted by the fused score and returned.

3.5. Embedding Model Evaluation & Upgrade

• Decision: Replace the default ChromaDB embedding function with a dedicated, explicitly configured multilingual model.
• Recommendation: Utilize intfloat/multilingual-e5-small (for lightweight CPU environments) or sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2. Both provide excellent English-Russian cross-lingual semantic alignment.
• Integration (DIP): Apply the Dependency Inversion Principle by injecting the embedding function (or an IEmbeddingProvider interface) into the ChromaStore constructor. This allows for seamless A/B testing of embedding models without touching the core storage logic.

4. Application to the Agent Architecture

• Vector Storage Agent (Database): This agent's responsibility shifts from "pure vector storage" to "Hybrid Storage Management." It coordinates the ChromaStore (Dense) and SQLiteStore (Sparse) implementations.
• AI Processor Agent: To maintain Single Responsibility (SRP), embedding generation can be shifted from the storage layer to the AI Processor Agent. The AI Processor generates the vector using an Ollama hosted embedding model and attaches it directly to the EnrichedNewsItemDTO. The Storage Agent simply stores the pre-calculated vector, drastically reducing the dependency weight of the storage module.

5. Next Steps for Implementation

1. Add sqlite3 FTS5 table initialization to the project scaffolding.
2. Refactor src/storage/base.py to segregate IStoreQuery and IStoreCommand.
3. Update ChromaStore to accept pre-calculated embeddings and utilize asyncio.to_thread.
4. Implement the RRF sorting algorithm in a new search_hybrid pipeline.
5. Update src/bot/handlers.py to route /latest through get_latest_by_category.