# LLD: AURA // Neural Visual Search System This Low-Level Design (LLD) document outlines the architecture, data models, exact match detection algorithms, API specifications, and frontend component layout of the **AURA Neural Visual Search System**. --- ## 1. Architectural Overview The system is split into a **FastAPI backend** running a local CLIP-based embedding engine and a **Vanilla HTML/CSS/JS frontend** featuring a glassmorphic user interface. ```mermaid graph TD UI[Frontend Client] -->|HTTP POST Image/Text| API[FastAPI App] API -->|Raw Text/Image| CLIP[CLIP Embedder] CLIP -->|512-dim Normalized Embedding| API API -->|Vector + Query Params| DB[In-Memory Vector Store] DB -->|Matches & Scores| API API -->|Classified Results exact_match + related| UI ``` --- ## 2. Component Specifications ### 2.1 Backend Modules #### `models.py` // `CLIPEmbedder` - **Model**: `openai/clip-vit-base-patch32` - **Execution Target**: Auto-detects `cuda` (if available), falls back to `cpu`. - **Image Embeddings**: Normalizes and runs image pixel values through the CLIP Vision Transformer. - **Text Embeddings**: Standardizes input strings, applies sub-word tokenization, and computes CLIP Text Transformer embeddings. - **Normalizing Vector Output**: Normalizes outputs to unit length ($L_2$ norm = 1.0) so that dot-product searches compute exact Cosine Similarity: $$\text{Cosine Similarity} = \vec{A} \cdot \vec{B}$$ #### `vector_store.py` // `InMemoryVectorEngine` & `VectorStore` - **Data Catalog**: Serialized as JSON list (`catalog.json`) containing metadata: - `id` (e.g. `prod-15970`) - `title` - `desc` - `price` - `category` - `image_url` - **Embeddings Store**: Serialized as a `.npy` NumPy array (`embeddings.npy`) of dimensions $(N, 512)$. - **Search Execution**: - Computes the dot product of the search vector with all stored product vectors. - Sorts indices in descending order. - Returns top $K$ results containing metadata and corresponding cosine similarity score. --- ## 3. Exact Product Match Classifier Logic The search classifier differentiates between an **Exact Match** and **Related Recommendations**. ```mermaid flowchart TD Start([Receive Query]) --> QueryType{Query Type?} QueryType -->|Text Query: q| TextExact{Exact Title/ID Match?} TextExact -->|Yes| SetExactText[exact_match = Product
score = Vector Score / 1.0] TextExact -->|No| SetExactNull[exact_match = null] QueryType -->|Image Upload| VectorSearch[Perform Vector Search] VectorSearch --> ImageScore{Top Similarity >= 0.92?} ImageScore -->|Yes| SetExactImage[exact_match = Top Product
score = Similarity Score] ImageScore -->|No| SetExactNull SetExactText --> FilterRelated[Filter exact_match ID from related_products] SetExactImage --> FilterRelated SetExactNull --> FilterRelated FilterRelated --> ReturnJSON([Return exact_match & related_products]) ``` ### 3.1 Text Search Exact Matching 1. The search query `q` is stripped of leading/trailing spaces and lowercased. 2. The database is checked for a product where: `q_clean == product["title"].strip().lower()` or `q_clean == product["id"].strip().lower()`. 3. If matched, it is returned as `exact_match`. If it is present in the vector search results, its calculated score is returned; otherwise, it defaults to a score of `1.0`. ### 3.2 Visual Search Exact Matching 1. The upload image is processed, and its unit-normalized embedding is calculated. 2. The vector index retrieves the top results. 3. If the highest cosine similarity score is $\ge 0.92$ (calibrated for matching identical or slightly transformed images), the top match is classified as the `exact_match`. 4. Otherwise, no exact match is returned. ### 3.3 Related Products Resolution 1. Vector search is executed with $K = 7$ to find similar items. 2. If an `exact_match` was resolved (via text or image check), its corresponding ID is filtered out of the results array. 3. The remaining array is sliced to return the top 6 `related_products`. --- ## 4. API Endpoints ### 4.1 Search Endpoint: `/api/search` - **Method**: `POST` - **Content-Type**: `multipart/form-data` - **Parameters**: - `q` (string, optional) - `file` (binary stream, optional) - **Response Format (`application/json`)**: ```json { "exact_match": { "product": { "id": "prod-15970", "title": "Turtle Check Men Navy Blue Shirt", "desc": "Navy Blue Shirts for Men. Designed for casual wear.", "price": 64.0, "category": "Apparel", "image_url": "/static/images/prod-15970.png" }, "score": 0.9998 }, "related_products": [ { "product": { ... }, "score": 0.8431 } ] } ``` ### 4.2 Similarity Endpoint: `/api/search/similar` - **Method**: `POST` - **Content-Type**: `application/json` - **Payload**: ```json { "product_id": "prod-15970" } ``` - **Response**: Array of similar products, excluding the queried product itself. --- ## 5. Frontend Layout & CSS Styling The page layout consists of a responsive container structured with CSS grids and glassmorphism panels. ### 5.1 Results Section Layout - **Exact Match Area (`#exact-match-section`)**: - Border: Dashed accent border with soft glow (`hsl(263, 70%, 50%)`). - Active State: Shows up to 1 card representing the exact match (enhanced with green `Exact Match` badge). - Empty State: Renders a premium banner (🕵️‍♂️ *No matching products found*) informing the user that no exact match is available in the database. - **Related Recommendations Grid (`#related-products-grid`)**: - CSS Grid structure: `repeat(auto-fill, minmax(260px, 1fr))`. - Header: *Related Products You Might Like*. - Renders the remaining 6 visual matches.