jdZddlZddlZddlmZddlmZddlmZddl m Z m Z eje ZdZdZid eefd efd efd efd efdefdeefdefdeeefdeefdeefdefdefdefdefdefZej-Dchc]\}}e|vse|vr|c}}Zd&dededefdZdededeeeffdZd edefd!Zdedefd"Zd#ed$ededzfd%Zycc}}w)'aProposes a field map for an unknown product API from one sample record. The proposal is never trusted: every path is executed against the sample before it is returned, so a merchant is never shown a mapping that would fail at sync time. Nothing is stored here -- the caller approves a map before it is saved. This runs once, on a source's first sync, and its result is cached in product_sources.config.fields. Every sync after that executes the stored map directly with zero model calls -- if inference ran per product, the mapping could drift between products, content_hash would drift with it, and the whole catalog would re-embed nightly for no reason. N)settings)client)FIELD_MAP_PROPOSAL_PROMPT) FieldMapError resolve_path) external_idtitle descriptionbrand raw_category image_urlpricediscount_percentage availabilityskutags related_ids cross_sellsupsellsrating review_count) url_templatecurrencyrr r r r r rrrrrrrrrrrecord max_value_lenreturnci}|xsijD]l\}}|}t|tr|d|}n4t|tr|dd}nt|trt|dd}t |j |d||<n|S)zReduce a record to shape -- keys, types, one truncated example each. A real product record can be tens of kilobytes of description prose; that teaches the model nothing about structure and costs money on every token. N)typeexample)items isinstancestrlistdictr __name__)rrsummarykeyvaluer!s E/var/www/html/strategist-ai/app/services/catalog/fieldmap_proposer.pysummarise_recordr,Bs G|**,J U eS !N]+G t $BQiG t $5k"1oG $U 4 4I J Nproposalsamplec ig}}|xsijD]\}}|tvr|j| t||}||j|At j |}|f|tvxrt|t}|s t||s>tjd|||t|j|j||||<|t|fS#t$r|j|YwxYw)aExecute every proposed path against the sample; keep only what resolves AND fits the target's storage. A path resolving is not enough: on one real API "$.reviews" resolved cleanly to a list of review objects that was proposed for review_count, an INT column, and psycopg2 could not adapt it. Nothing the model says is trusted -- an unknown target name, a malformed path, a path that finds nothing, or a value of the wrong type is dropped rather than stored. z0Dropping field map %r -> %r: expected %s, got %s)r" TARGET_FIELDSappendrr FIELD_TYPESget_NUMERIC_TARGETSr#boolloggerwarningr r'sorted) r.r/keptdroppedtargetpathr*expected is_bad_bools r+validate_proposalr@Us'D!R..0   & NN6 "   .E = NN6 " ??6*   $44PE49PK*UH"=FD(DK,@,@v&V 12   '  NN6 "  s C++DDr(c vtjjjtj ddidt jdjtddtj|tdg }tj|jd jj S) Nr json_objectsystemz, )targets)rolecontentuser)default)modelresponse_formatmessagesr)rchat completionscreater LLM_MODELrformatjoinr1jsondumpsr$loadschoicesmessagerF)r(responses r+ _call_llmrX|s{{&&--  /188=13 4 7C(H I  . H ::h&&q)1199 ::r-c tt|}t |t stjdi}t||\}}||dS#t$rtj ddi}Y_wxYw)zAsk the model to map one sample record, then verify every answer. A model outage or a malformed response must leave the source connected and mappable by hand -- never break the sync, and never store an unvalidated guess. zField-map proposal failedT)exc_infoz.Field-map proposal was not an object; ignoring)fieldsr;) rXr, Exceptionr7errorr#r&r8r@)r/rawr:r;s r+propose_field_mapr_sw(01 c4 GH%c62MD'w //  04 @sA"A65A6base_urlr[ch|rd|xsivry|jddjd}|dS)aDerive a per-product URL from a collection endpoint. A REST collection almost always addresses one member by id, so ".../products" plus an id mapping yields ".../products/{external_id}". Without an id mapping there is nothing to substitute, and guessing would give every product a broken link -- so this returns None and those products are stored with a null URL instead. rN?r/z/{external_id})splitrstrip)r`r[roots r+infer_url_templatergsA }V\r: >># q ! ( ( -DV# $$r-)P)__doc__rRloggingapp.core.configrapp.core.llm_clientrapp.core.promptsrapp.services.catalog.fieldmaprr getLoggerr'r7r1 NOT_MODELLEDr$intfloatr%r3r"r5r&r,tupler@rXr_rg)r<typess00r+rus  $&6E   8 $ , C: C6C6 C6  C6  C6  C:C6 C,C< C<C6 D7D7D7 D7! ,!, 1 1 3vu e|u~  T#t&$!$!d$!uT4Z7H$!N ;t ; ;0d0t0* % %d %sTz %GsC&