# Product Catalogue & Recommendations — Integration Reference How a merchant's catalogue gets in, what happens to it, and where every piece is stored. --- ## 1. Shopify integration ### 1.1 The endpoints | Method | Path | Called by | Purpose | |---|---|---|---| | `GET` | `/api/shopify/install?shop=.myshopify.com` | Merchant's browser | Starts OAuth. Validates the shop domain, stores a state nonce in Redis, redirects to Shopify's consent screen | | `GET` | `/api/shopify/callback` | Shopify (browser redirect) | Verifies HMAC, checks the state nonce, exchanges the code for an offline access token, stores it encrypted, records the integration | | `GET` | `/api/shopify/status?tenant_id=` | Merchant UI | Whether Shopify is connected and to which shop | Routes: `app/api/shopify.py`. Logic: `app/services/integrations/shopify_oauth.py`. ### 1.2 The OAuth flow ``` merchant clicks Connect -> GET /api/shopify/install?shop=acme.myshopify.com -> redirect to https://acme.myshopify.com/admin/oauth/authorize?...&state= -> merchant approves -> Shopify redirects the browser to GET /api/shopify/callback?code=...&hmac=...&state=... -> verify HMAC, verify state, POST to Shopify for the access token -> store encrypted, write the integration rows -> redirect back to the app ``` **Offline token**, not online: the sync runs on a schedule the merchant is not present for, so a session-bound token would be useless. **HMAC verification happens before anything else is trusted.** The state nonce is single-use and lives in Redis, so a replayed callback fails. ### 1.3 Configuration | Setting | Where | Notes | |---|---|---| | `SHOPIFY_CLIENT_ID` | `.env` | Should move to `platform_configs`, see §4.2 | | `SHOPIFY_CLIENT_SECRET` | `.env` | Same | | `PUBLIC_BASE_URL` | `.env` | The callback is `{PUBLIC_BASE_URL}/api/shopify/callback` and must match the Shopify app's "Allowed redirection URL" character for character | | `SOURCE_CREDENTIALS_KEY` | `.env` | Fernet key that encrypts stored tokens. Not in any database | ### 1.4 What is fetched Admin GraphQL API `2026-07`: id, title, description, vendor, handle, `onlineStoreUrl`, featured image, category, product type, tags, collections, status, variants (price, compare-at, availability, options) and the Search & Discovery `complementary_products` metafield. **A product not published to the Online Store sales channel has no `onlineStoreUrl`.** It imports with a null URL, is flagged incomplete, and is excluded from recommendations. This is the most common surprise after a first sync. --- ## 2. The other two sources Shopify is one of three ways in. All converge on the same normaliser and the same table. | Source | Connect | Fetch | |---|---|---| | Shopify | `GET /api/shopify/install` | `POST /catalog/sync` | | HTTP product API | `POST /sources/http` | `POST /catalog/sync` | | CSV upload | — | `POST /catalog/import/csv` | `GET /sources` lists what is connected; `DELETE /sources/{kind}/{external_ref}` disconnects. **A CSV is not a syncable source** — the file only exists at upload time, so it is never registered in `product_sources` and `/catalog/sync` ignores it. **Currency is never guessed.** Shopify reports it; the HTTP source and CSV upload must declare it. A fallback was tried once and mislabelled 194 USD products as INR. --- ## 3. The pipeline ``` POST /catalog/sync fetch, normalise, upsert (per source) POST /catalog/import/csv parse, normalise, upsert (per file) | POST /catalog/enrich extract attributes with an LLM, cached on content_hash | POST /catalog/pair embed, score, build the graph | POST /recommendations/event serve up to 3 cards to a shopper ``` None of these run on a schedule. There is no cron, no queue and no Shopify webhook — a catalogue is stale until someone calls `/catalog/sync` again. ### 3.1 Merchant-facing reads | Method | Path | Purpose | |---|---|---| | `GET` | `/catalog/categories` | Category list with counts | | `GET` | `/catalog/products` | Paged, filterable by category | | `GET` | `/catalog/products/{key}` | One product | | `GET` | `/catalog/products/{key}/pairings` | Its pairings by type, with scores and reasons | | `GET` | `/catalog/pairings/pending` | The approval queue | | `POST` | `/catalog/pairings/decide` | Approve or reject, batched | | `GET` | `/catalog/admin` | Internal admin page | | `GET` | `/catalog/demo` | Chat and Connect-Shopify demo page | | `GET` | `/catalog/sample.csv` | CSV template | --- ## 4. Where everything is stored Two databases, split on a single principle: **the master database knows how to reach a merchant's data; the tenant database holds the data.** ### 4.1 `galaxiq_master` — connections and credentials | Table | Holds | |---|---| | `product_sources` | The working record per connected source: `tenant_id`, `kind`, `external_ref`, `config`, `credentials_encrypted`, `status`, `connected_at`, `last_synced_at` | | `integrations` | Platform-standard row: `provider`, `authType`, `status` | | `integration_connect_tracker` | Audit trail of connect attempts | | `platform_configs` | Per-platform OAuth client credentials | `credentials_encrypted` is a Fernet blob. **The key lives in `.env`, never in the database**, so a database dump alone does not expose any merchant's token. `external_ref` identifies the connection — the shop domain for Shopify, the host for an HTTP API, the filename for a CSV batch. ### 4.2 Two gaps worth knowing **`platform_configs` has no `shopify` row.** Thirteen platforms are configured there; Shopify is not, so its client credentials are read from `.env` instead. **`integration_connect_tracker` has no Shopify rows**, so platform-level connection-health reporting does not see Shopify connects. Neither breaks the connector. Both are the work of making Shopify a first-class citizen alongside the other integrations. ### 4.3 `galaxiq_tenants` — the catalogue, one schema per tenant Schema name is the tenant id, e.g. `org_8c32bf3e-…`. | Table | Holds | |---|---| | `strategist_products` | The catalogue. Every source merged, each row tagged `source_kind` + `source_ref` | | `strategist_product_neighbors` | The pairing graph: `anchor_key`, `neighbor_key`, `pair_type`, `score`, `confidence`, `source`, `reasons` | | `strategist_pairing_decisions` | Merchant approvals and rejections. **Never written by the pairing job** | | `strategist_product_embeddings` | Cached vectors keyed on `content_hash` | **Isolation is structural.** A tenant's catalogue lives in its own schema, so one tenant cannot read another's — it is not a `WHERE` clause anyone can forget. ### 4.4 The scoping rule that keeps sources apart Every product row carries `source_kind` and `source_ref`, and **every delete is scoped to that pair**. A Shopify sync only ever reaps Shopify rows for that shop. Without it, a website re-crawl would delete a synced Shopify catalogue. That was a real defect, caught before release, and the guard is now the load-bearing rule of the whole ingestion design. ### 4.5 Product URL vs source URL Two different things, stored in two different places: | | Where | What | |---|---|---| | Source URL | `galaxiq_master.product_sources.config` | Where to fetch from — a base URL and a template | | Product URL | `.strategist_products.product_url` | Where a shopper lands — one per product | The master row holds the recipe; the tenant rows hold the results. --- ## 5. Recommendations `POST /recommendations/event` is the only shopper-facing endpoint. ```json {"tenant_id": "...", "visitor_id": "...", "event": "dwell", "page": {"url": "https://shop.example/products/x", "dwell_seconds": 60}} ``` Returns up to three cards, each with `pair_type` and `score`, or `{"recommend": false, "reason": "..."}` when there is nothing to show. **It never returns an error** — a suggestion nobody asked for must not break a shopper's page. Three pairing types are computed: `similar` (a substitute), `complement` (something that goes with it) and `upsell` (a pricier version). **Only `similar` and `complement` are served.** Nothing unapproved, rejected, out of stock or incomplete is ever served, and that is evaluated at request time — so a merchant's rejection takes effect immediately, without re-running the pairing job. --- ## 6. Security notes - Tokens are encrypted at rest with a key held outside the database. - Merchant-supplied URLs are HTTPS-only and checked against their resolved IP, so a hostname cannot be pointed at an internal address. - No endpoint in this service is authenticated. It must not be exposed publicly without a gateway in front.