# Strategist AI - API Documentation

> [!IMPORTANT]
> **Production Configuration**: Hardcoded secrets have been removed for security. You must provide `DB_PASSWORD`, `OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, and `FIRESTORE_PROJECT` via environment variables or a `.env` file.

Strategist AI is a multi-tenant RAG (Retrieval-Augmented Generation) system designed for strategic analysis, automated support ticketing, and knowledge management. It features **Intelligent Resource Quotas**, **Redis-cached plan mappings**, and **real-time administrative notifications** for plan compliance.

---

## 🏗️ Multi-Tenant Architecture & Quotas

The system enforces strict resource isolation and intelligent quota limits by bridging with the `galaxiq_master` environment.

### 📊 Resource Tracking & Plans
Each tenant's limits are dynamically fetched from the Master DB and mapped to 4 core categories using a cached LLM analyzer:
1.  **AI Tokens**: Total token consumption across all models.
2.  **Tickets**: Maximum number of support tickets allowed.
3.  **Vector DB Space**: Knowledge base storage limit (in MB).
4.  **Conversations**: Total AI interaction count.

### ⚡ Performance Optimization
- **Redis Caching**: Resource mappings and plan limits are cached in Redis (10-minute TTL) for sub-millisecond enforcement checks.
- **Quota Headers**: Every API response includes current usage vs. limits.

### 🔔 Administrative Notifications
When a tenant reaches their allocated limit, the system automatically triggers an external alert via the **GalaxiQ Notification API**:
- **Endpoint**: `POST https://dev-api.galaxiq.ai/api/admin/notification`
- **Triggers**: Hit on any of the 4 core metrics.
- **Payload**: Includes the specific user ID and resource-type description.

---

## 🚀 Core Endpoints

### 1. System Setup
#### **POST** `/bootstrap`
Initializes a new tenant-specific PostgreSQL schema, sets up all required tables, and **dynamically extracts brand persona (tone/voice)** from the latest brand analysis to configure the AI's personality.
- **Query Parameters**:
    - `tenant_id`: `string` (Unique ID for the organization)
- **Response**:
```json
{
  "status": "success",
  "message": "Tenant org_123 setup complete."
}
```

---

### 2. Knowledge Ingestion
These endpoints process diverse content sources, generate high-dimensional embeddings, and prepare the AI knowledge base.

#### **POST** `/ingest/files`
Extracts and indexes text from PDF, DOCX, and TXT files. **Automatic Replacement**: If a file with an identical name is uploaded, the previous version is automatically deleted from both the database and Azure storage before the new one is ingested.
- **Payload (Form-Data)**:
    - `tenant_id`: `string`
    - `user_id`: `string` (Optional, used for resource attribution)
    - `files`: `[file_binary]`
- **Response Example**:
```json
{
  "status": "success",
  "message": "Ingested 2 files into org_123.",
  "summary": "The uploaded documents provide a technical overview of...",
  "ingested_date": "2026-04-27",
  "ingested_time": "13:05:10",
  "created_at": "2026-04-27T13:05:10.123456Z"
}
```

#### **POST** `/ingest/website`
Crawls a URL (depth 1, up to 20 pages), extracts text, identifies brand colors, and saves site metadata. 
- **Restricted Scope**: The crawler only follows links that are sub-paths of the starting URL (e.g., crawling `/products` will NOT visit `/about`).
- **Automatic Replacement**: If the same URL (or sub-page) is re-ingested, all previous data for that specific path is cleared first.
- **Payload (Form-Data)**:
    - `tenant_id`: `string`
    - `user_id`: `string` (Optional, used for resource attribution)
    - `url`: `https://example.com`
- **Response Example**:
```json
{
  "status": "success", 
  "message": "Ingested website content from https://example.com into org_123.",
  "summary": "Example Corp is a leader in renewable energy...",
  "metadata": {
    "source": "https://example.com",
    "urls_visited": ["https://example.com/", "https://example.com/about"],
    "colors": ["#0055aa", "#ffffff"],
    "top_images": [{"url": "https://example.com/logo.png", "label": "Example Corp Logo"}],
    "ingested_date": "2026-04-27",
    "ingested_time": "13:07:45",
    "created_at": "2026-04-27T13:07:45.654321Z"
  },
  "ingested_date": "2026-04-27",
  "ingested_time": "13:07:45",
  "created_at": "2026-04-27T13:07:45.654321Z"
}
```

#### **POST** `/ingest/text`
Directly indexes text content. **Automatic Replacement**: If text with an identical title is ingested, the previous version is replaced. If text is formatted as `Title: Your Title Content: Your Content`, it extracts the explicit title prefix. Otherwise, an LLM generates a title.
- **Payload (Form-Data)**:
    - `tenant_id`: `string`
    - `text_content`: `string`
- **Response Example**:
```json
{
  "status": "success", 
  "message": "Ingested text content into org_123.",
  "summary": "This document outlines the internal security policies for Q1...",
  "ingested_date": "2026-04-27",
  "ingested_time": "13:10:20",
  "created_at": "2026-04-27T13:10:20.987654Z"
}
```

#### **DELETE** `/ingest/source`
Granularly removes data from the knowledge base using its unique `source_id`. This perfectly isolates files with identical names.
- **Query Parameters**:
    - `tenant_id`: `string` 
    - `source_id`: `string` (The UUID returned when listing sources)
    - `source_name`: `string` (Optional, the original filename needed to delete from Azure Blob Storage)
- **Response Example**:
```json
{
  "status": "success",
  "message": "Deleted 12 chunks for source_id '1bd4660b-8b44-4cfe-bd8e-3a5bfdded7b0' from org_123 and Azure storage.",
  "deleted_count": 12
}
```

---

### 3. AI Chat & Threading
Intelligent RAG chat with persistent memory and multi-role interaction.

#### **POST** `/chat`
Primary interaction loop. Automatically retrieves context from the knowledge base, manages Firestore threads, handles automated ticketing, and **dynamically adopts the tenant's brand persona (tone/voice)**. 
- **Behavioral Constraints**:
    - **Strict Context**: Only answers from provided knowledge.
    - **Concise Response**: Responses are limited to **6-7 lines** maximum for extreme crispness.
    - **Resource Integration**: Automatically extracts relevant URLs and image links from the specific sub-pages of the provided context to assist user navigation. Each answer is backed by deep-links where possible.
    - **Native Language Adaptation**: If a `location` is provided (or persisted), the AI automatically adapts its professional tone, vocabulary, and spelling to the local business dialect (e.g., Business Australian English for "Australia").
- **Payload (Form-Data)**:
    - `tenant_id`: `string` (Required)
    - `query`: `string` (Required)
    - `thread_id`: `string` (Default: `default`)
    - `user_id`: `string` (Required for isolation and quota tracking)
    - `user_name`: `string` (Optional, used for ticketing)
    - `role`: `string` (`user` or `admin`)
    - `location`: `string` (Optional. sets the native business dialect)
- **Response (Success)**:
```json
{
  "status": "success",
  "response": "Hello Nitish, I've checked the manuals...",
  "resources": [
    {"url": "...", "image": "...", "label": "..."}
  ],
  "thread_id": "thread_abc",
  "user_id": "user_456"
}
```
- **Response (Quota Hit)**:
When a limit is reached, the AI returns a `status: error` but includes the user-friendly explanation directly in the `response` field for UI display.
```json
{
  "status": "error",
  "response": "This service is currently unavailable. Please try again in a little while.",
  "resources": [],
  "thread_id": "thread_abc",
  "user_id": "user_456"
}
```

#### **Native Language Examples**
| Input `location` | AI Response Tone/Dialect |
| :--- | :--- |
| `Australia` | Uses Australian business English (e.g., "G'day", "reckon", "standardised"). |
| `United Kingdom` | Uses British business English (e.g., "Cheers", "organised", "programme"). |
| `India` | Uses professional Indian business English (e.g., "Kindly revert", "Please do the needful"). |

#### **GET** `/ingest/files` | `/ingest/websites` | `/ingest/text`
Retrieves a categorized list of all resources currently in the knowledge base, including their distinct UUIDs and split ingestion timestamps.
- **Query Parameters**: `tenant_id`
- **Response Example**:
```json
{
  "status": "success",
  "files": [
    {
      "source_id": "1bd4660b-8b44-4cfe-bd8e-3a5bfdded7b0",
      "source": "contract_v1.pdf",
      "ingested_date": "2026-04-22",
      "ingested_time": "14:10:30",
      "created_at": "2026-04-22T14:10:30Z",
      "metadata": {"ingestion_type": "file", "source_id": "1bd4660b-...", "source": "contract_v1.pdf"}
    }
  ]
}
```

#### **GET** `/chat/threads`
Lists all active conversational threads for a tenant.
- **Query Parameters**: 
    - `tenant_id`: `string`
    - `user_id`: `string` (Optional. If provided, returns only that user's threads. If omitted, returns all tenant threads for admins.)
- **Response Example**:
```json
{
  "status": "success",
  "count": 1,
  "threads": [
    {
      "thread_id": "thread_abc_789",
      "user_name": "Nitish",
      "summary": "User inquiring about scraper maintenance.",
      "created_at": "2026-02-24T10:00:00Z"
    }
  ]
}
```

#### **GET** `/chat/history`
Retrieves the full message log for a specific thread, correctly identifying `user`, `admin`, and `assistant` roles.
- **Token Usage**: Verified that all LLM calls correctly report and **persist** token consumption in the `strategist_llm_usage` table. Accessible via `GET /analytics/usage`.
- **Query Parameters**: `tenant_id`, `thread_id`, `user_id`
- **Response Example**:
```json
{
  "status": "success",
  "thread_id": "thread_abc_789",
  "messages": [
    {"role": "user", "content": "How do I fix the scraper?"},
    {"role": "admin", "content": "I have updated the maintenance logs."},
    {"role": "assistant", "content": "According to the admin and the manual..."}
  ]
}
```

#### **POST** `/chat/takeover`
Toggles the **Human Takeover** mode for a specific conversation. When active, the AI will stop responding to user queries via the REST API, allowing a human agent to intervene.
- **Payload (Form-Data)**:
    - `tenant_id`: `string`
    - `thread_id`: `string`
    - `user_id`: `string` (Optional for legacy, but required for new threads)
    - `status`: `boolean` (`true` to activate human takeover, `false` to return to AI)
- **Response Example**:
```json
{
  "status": "success",
  "takeover_active": true,
  "message": "Human takeover activated for thread_abc_789"
}
```

---

#### **WS** `/ws/{tenant_id}/{thread_id}`
Dedicated stream for a specific conversation. Both user and admin connect here for real-time messaging.
- **Role**: `user` or `admin`.
- **Logic**: Handles sub-millisecond bidirectional chat and `takeover_started`/`takeover_ended` system events.

#### **WS** `/ws/{tenant_id}/conversation_list`
Discovery stream for the Admin Dashboard.
- **Events**: `new_thread_created`, `intervention_requested`, `takeover_started`, `takeover_ended`.
- **Purpose**: Instantly update the inbox list when new users start chatting or request help.

#### **WS** `/ws/{tenant_id}/tickets`
Real-time synchronization for the Support Tickets page.
- **Events**: `new_ticket_created`, `ticket_updated`, `ticket_status_updated`.
- **Purpose**: Instantly update the tickets table when AI or Admins modify ticket data.

---

## 🛠️ Frontend Integration
For a detailed implementation guide on how to bind these WebSockets and manage the Human-AI handover state, refer to **[FRONTEND_INTEGRATION.md](./FRONTEND_INTEGRATION.md)**.

---

### 5. Real-Time Event Payloads
The system sends "System" events over WebSockets to ensure smooth UI transitions.

#### **New Thread Created**
Sent to `/ws/{tenant_id}/dashboard`.
```json
{
  "type": "system",
  "event": "new_thread_created",
  "message": "A new conversation has started",
  "data": {
    "thread_id": "thread_abc_123",
    "user_id": "user_789",
    "user_name": "Anonymous",
    "summary": "",
    "takeover_active": false,
    "created_at": "2026-03-01T12:00:00Z",
    "updated_at": "2026-03-01T12:00:00Z",
    "last_message_preview": "Hello, can you help me with..."
  }
}
```

#### **Takeover Status Change**
Sent to both `/ws/{tenant_id}/dashboard` and `/ws/{tenant_id}/{thread_id}`.
```json
{
  "type": "system",
  "event": "takeover_started",
  "message": "Human agent has joined the chat",
  "data": {
    "thread_id": "thread_abc_123",
    "user_id": "user_789",
    "takeover_active": true
  }
}
```
> [!NOTE]
> When received on the specific thread WebSocket, the `thread_id` can be used for verification. When received on the Dashboard WebSocket, it is essential for the UI to know which conversation's status has changed.

---

### 6. Retrieval & Analytics

#### **GET** `/summary/{type}`
Fetches the latest consolidated summary for a specific ingestion type.
- **Path Parameters**: `type` (`files`, `website`, or `text`)
- **Query Parameters**: `tenant_id`
- **Response Example**:
```json
{
  "tenant_id": "org_123",
  "type": "website",
  "summary": "This website focuses on sustainable architecture and...",
  "ingested_date": "2026-04-27",
  "ingested_time": "13:15:00",
  "created_at": "2026-04-27T13:15:00.000000Z"
}
```

#### **GET** `/analytics/usage`
Retrieves the history of LLM token consumption for a tenant, broken down by feature and model.
- **Query Parameters**: 
    - `tenant_id`: `string`
    - `limit`: `integer` (Default: 100)
- **Response Example**:
```json
{
  "status": "success",
  "tenant_id": "org_123",
  "count": 2,
  "usage": [
    {
      "feature_name": "Chat",
      "model_name": "gpt-5.2-chat",
      "prompt_tokens": 415,
      "completion_tokens": 267,
      "total_tokens": 682,
      "created_at": "2026-02-24T15:34:29Z"
    },
    {
      "feature_name": "Embedding",
      "model_name": "text-embedding-3-large",
      "prompt_tokens": 7,
      "completion_tokens": 0,
      "total_tokens": 7,
      "created_at": "2026-02-24T15:34:09Z"
    }
  ]
}
```

#### **GET** `/analytics/refresh`
Triggers a multi-stage analytics pipeline:
1.  **Discovery**: Scans Firestore for new chat threads.
2.  **AI Analysis**: Enriches new threads with sentiment, intent, and resolution signals.
3.  **dbt Transformation**: Rebuilds the analytics Mart layer using dbt for precise, standardized KPI calculation.
4.  **Consolidation**: Returns a unified dashboard of 30+ KPIs across 6 business dimensions.
- **Query Parameters**:
    - `tenant_id`: `string`
- **Response Example**:
```json
{
  "status": "success",
  "data": {
    "tenant_id": "org_123",
    "metrics": {
      "general": {
        "total_conversations": 45,
        "avg_sentiment": 0.72,
        "bot_containment_rate": 88.5
      },
      "intent_and_demand": [
        {"intent": "support", "count": 20},
        {"intent": "pricing", "count": 12}
      ],
      "voice_of_customer": {
        "positive_points": "• Fast response times\n• Friendly greeting",
        "key_concerns": "• Issue with payment integration\n• High latency in dashboard",
        "pain_point_frequency": 5,
        "feature_request_volume": 3
      },
      "total_ai_responses": 64
    }
  }
}
```

> [!TIP]
> **Understanding the Metrics**: For a detailed, step-by-step breakdown of how each KPI is calculated (including weighting and logic formulas), refer to [KPI_LOGIC.md](./analytics/KPI_LOGIC.md).

## Storage Architecture & Sync Logic

The system utilizes a hybrid storage strategy to ensure both search performance (PostgreSQL) and data persistence (Azure).

### Multi-Tenant Strategy
Storage is strictly isolated by `tenant_id`:
- **Database**: Each tenant has a dedicated PostgreSQL **schema** (e.g., `org_123`).
- **Azure Blob Storage**: Files are stored in a common container but isolated via **prefix paths**: `tenant/{tenant_id}/files/{filename}`.

### Resource Storage Workflow
| Resource Type | DB Table | Persistent Storage (Azure) | Metadata Key |
| :--- | :--- | :--- | :--- |
| **Files** | `strategist_knowledge_base` | **Yes** (Blob Storage) | `ingestion_type: "file"` |
| **Websites** | `strategist_knowledge_base` | No (Scraped Content Only) | `ingestion_type: "website"` |
| **Manual Text** | `strategist_knowledge_base` | No (Raw Text Only) | `ingestion_type: "text"` |

### Deletion Logic (Synchronized)
When `DELETE /ingest/source` is called, the system performs a dual, highly targeted cleanup using `source_id`:
1.  **PostgreSQL**: All vector chunks (3072D embeddings) exactly matching the `source_id` are purged from the AI's knowledge base.
2.  **Azure**: If a `source_name` is provided for a file, the specific blob (`{source_id}_{source_name}`) is permanently deleted from Azure Storage, ensuring identical filenames never collide.

---

## Technical Documentation & Transparency
- **KPI Logic**: [KPI_LOGIC.md](./analytics/KPI_LOGIC.md)
- **Database Schema**: [database.py](./app/services/database.py)
- **Azure Storage Service**: [storage.py](./app/services/storage.py)

---

#### **GET** `/analytics/general`
Simplified endpoint for UI dashboards. Returns Top KPIs (with period-over-period trends) and time-series arrays for visual rendering. Supports both relative time ranges and explicit date-based filtering.
- **Query Parameters**:
    - `tenant_id`: `string` (Required)
    - `time_range`: `string` (Optional, e.g., `1h`, `24h`, `7d`, `30d`, `all`. Default: `all`)
    - `start_date`: `string` (Optional, `YYYY-MM-DD`. Overrides `time_range` if provided)
    - `end_date`: `string` (Optional, `YYYY-MM-DD`. Defaults to today if `start_date` is provided)
- **Response Example**:
```json
{
  "status": "success",
  "tenant_id": "test_ticketing",
  "data": {
    "top_kpis": {
      "total_conversations": {
        "value": 45,
        "trend": "+11.01%",
        "trend_direction": "up"
      },
      "avg_response_time": {
        "value": "1.2s",
        "trend": "+11.01%",
        "trend_direction": "up"
      },
      "resolution_rate": {
        "value": "88.5%",
        "trend": "-11.01%",
        "trend_direction": "down"
      },
      "user_satisfaction": {
        "value": "4.2/5",
        "trend": "+11.01%",
        "trend_direction": "up"
      },
      "total_ai_responses": {
        "value": 64,
        "trend": "Live",
        "trend_direction": "flat"
      }
    },
    "charts": {
      "total_users": {
        "labels": ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"],
        "datasets": [
          {
            "label": "2026",
            "data": [10, 15, 20, 30, 0, 0, 0, 0, 0, 0, 0, 0]
          },
          {
            "label": "2025",
            "data": [5, 8, 12, 18, 20, 25, 28, 30, 32, 35, 40, 45]
          }
        ]
      },
      "conversation_channels": {
        "labels": ["Website Chat"],
        "datasets": [
          {
            "data": [100.0],
            "raw_counts": [45]
          }
        ]
      },
      "response_time": {
        "labels": ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"],
        "datasets": [
          {
            "label": "2026",
            "data": [1.5, 1.3, 1.2, 1.2, 0, 0, 0, 0, 0, 0, 0, 0]
          }
        ]
      }
    }
  }
}
```

#### **GET** `/tickets`
Lists all support tickets generated by the AI, including explicitly requested contact details natively.
- **Query Parameters**: `tenant_id`
- **Response Example**:
```json
{
  "status": "success",
  "count": 1,
  "status_counts": {
    "Open": 1,
    "In Progress": 2,
    "Ongoing": 1,
    "Resolved": 0
  },
  "tickets": [
    {
      "id": 1,
      "ticket_id": "TICK-A1B2C3D4",
      "user_name": "Nitish",
      "heading": "Website Down",
      "content": "The login button on the homepage is completely broken and not functioning.",
      "priority": "High",
      "status": "Open",
      "email": "nitish@example.com",
      "contact_no": "555-0100",
      "contact_medium": "WhatsApp",
      "contact_details": null,
      "created_at": "2026-02-27T15:00:00Z",
      "thread_id": "thread_abc_789"
    }
  ]
}
```

#### **POST** `/tickets/{ticket_id}/status`
Updates the status of an existing ticket.
- **Path Parameters**: `ticket_id` (e.g., `TICK-A1B2C3D4`)
- **Payload (Form-Data)**:
    - `tenant_id`: `string`
    - `status`: `string` (Must be one of: `In Progress`, `Resolved`, `Ongoing`, `Open`)
- **Response Example**:
```json
{
  "status": "success",
  "message": "Ticket status updated to In Progress",
  "ticket_id": "TICK-A1B2C3D4",
  "new_status": "In Progress"
}
```

#### **POST** `/tickets/{ticket_id}/comments`
Allows an administrator or support agent to post a comment or status update on a specific ticket.
- **Path Parameters**: `ticket_id` (e.g., `TICK-A1B2C3D4`)
- **Payload (Form-Data)**:
    - `tenant_id`: `string`
    - `admin_name`: `string` (Optional, defaults to "Admin")
    - `comment`: `string`
- **Response Example**:
```json
{
  "status": "success",
  "message": "Comment added successfully",
  "comment": {
    "id": 1,
    "ticket_id": "TICK-A1B2C3D4",
    "admin_name": "Support Lead Sarah",
    "comment": "I have reviewed this issue and escalated it to the frontend team.",
    "created_at": "2026-02-27T04:48:43Z"
  }
}
```

#### **GET** `/tickets/{ticket_id}/comments`
Retrieves all comments chronologically for a specific ticket to display the full interaction history.
- **Path Parameters**: `ticket_id` (e.g., `TICK-A1B2C3D4`)
- **Query Parameters**: `tenant_id`
- **Response Example**:
```json
{
  "status": "success",
  "count": 1,
  "comments": [
    {
      "id": 1,
      "ticket_id": "TICK-A1B2C3D4",
      "admin_name": "Support Lead Sarah",
      "comment": "I have reviewed this issue and escalated it to the frontend team.",
      "created_at": "2026-02-27T04:48:43Z"
    }
  ]
}
```

---

## 🛠️ Key Systems

### Role Handling
The system distinguishes between **User** and **Admin** interactions. 
- Administrative inputs are prefixed with `[Admin]:` in the AI's short-term memory.
- This allows the AI to prioritize or acknowledge administrative instructions during a multi-party strategy session.

### Ticketing Workflow
The system employs a **one-ticket-per-session** architecture. If multiple issues are raised within the same conversation thread, the system automatically appends the information to the existing ticket and updates its heading and priority based on the latest interaction.

The AI is instructed to gather the following before generating/updating a ticket:
1.  **User Name** & **Issue Heading**
2.  **Priority Level** (Low/Medium/High)
3.  **Contact Medium** (e.g., Phone, Email, Slack)
4.  **Contact Details** (e.g., actual number or address)

### Knowledge Base Deletion
Data removal is "source-aware." When you delete a source (like `branding_guide.pdf`), the system identifies all vector chunks that originated from that specific file and purges them, while leaving other files in the knowledge base untouched.

---

## ⚙️ Technical Specs
- **Embeddings**: `text-embedding-3-large` (3072 dimensions)
- **Vector Search**: PostgreSQL `pgvector` with Cosine Similarity
- **Memory**: Persistent Firestore threading
- **Isolation**: Per-tenant PostgreSQL schemas for strict data privacy