jsf ddlmZmZmZmZmZmZmZmZm Z ddl m Z ddl m Z mZddlmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZddl m!Z!ddl"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(ddl)m*Z*dd l+m,Z,m-Z-dd l.m/Z/m0Z0dd l1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9dd l:m;Z;mZ>m?Z?m@Z@mAZAdd lBmCZCddlDmEZEddlFmGZGmHZHddlImJZJmKZKddlLmMZMmNZNddlOmPZPddlQmRZRmSZSddlTmUZUddlVZVddlWZWddlXZXddlYZYddlZZZddl[Z[ddl\Z\ddl]m]Z]m^Z^m_Z_eYjeaZbeZcecjddeefdZfecjdededededfdeedeeedeeede efdZgecjd edededfdeed!eedeeefd"Zhecjd#edededfdeed$eedeeefd%Ziecjd&edd'(fdeed)eed*eefd+Zkecjd,deed-eefd.Zmecjddeefd/Znecjd0deefd1Zoecjd deefd2Zpecjd3ededededfdeedeeedeeede efd4Zqecjd3deed)eefd5Zrecjd3d}deedeeefd6Zsecjd7edededfdeed)eedeefd8Ztecjd9 d~deed)eed:eefd;Zuecjd<deefd=Zvd>ewfd?Zxecjd<d@efdAZyecjdBedfdCezfdDZ{ecjdEedededFedededGedfdeedHeedIeedeedJeedKeedLeeefdMZ|ecjdNedfdeedeefdOZ}ecjdPdFedfdeedIeedeefdQZ~ecjdRdeefdSZecjdTededUedfdVeedeedWeedXeefdYZecjdTedfdVeedeefdZZecjd[ededfdVeedeed\eefd]Zecjd^eded_fdeed`efdaZecjdbdeefdcZecjddededf(eddg(eddh(fdeedieedjeedkeefdlZecjdmededededfdeedIeed\edeefdnZdoZdpezdeed>efdqZddsedtefduZecjdvedededrdfdeedweeedxedye fdzZecjd{ededredrdfdeedtedxedye fd|Zy)) APIRouterBody UploadFileFileForm HTTPExceptionQueryRequestResponse)RedirectResponse)ListOptional)bootstrap_tenantinsert_vector_datainsert_summaryget_latest_summary get_ticketsget_ticket_status_countsdelete_vector_data_by_source get_llm_usageget_knowledge_sourcesadd_ticket_commentget_ticket_commentsinsert_crawled_urlupdate_ticket_status get_ticket!delete_vector_data_by_url_subpathget_source_ids_by_name)storage_service) chunk_textget_embeddingsgenerate_summarygenerate_title)extract_text_from_url)get_chat_response)get_all_threadsget_chat_history)get_kpisget_general_kpis)DEFAULT_CATEGORYblob_urldelete_document download_link find_documentingest_documentslist_documents set_category)EMAIL_COPY_DEFAULTSFEATURE_DEFAULTS FEATURE_TOOLSPROVIDER_OPTIONSget_tool_settingsnormalise_tenant_idsave_tool_settings)QuotaExceededError enrich_tenant) VALID_EVENTSdecide)has_any_products sync_source) KIND_CRAWL get_sources)jobs)_run start_job)settingsN)datetime timedeltatimezonez /bootstrap tenant_idc jKtjd| t|dd|ddS#t$r>}tj d|d|d t d d d t |d d}~wt$r2}tj d|d|d t dd dd d}~wwxYww)Nz'Received bootstrap request for tenant: successzTenant z setup complete.statusmessagez)Validation error in bootstrap for tenant : Texc_infoerrorzValidation failed:  status_codedetailz'Error in bootstrap endpoint for tenant z*Internal server error during tenant setup.)loggerinfor ValueErrorrSrstr Exception)rIvees 0/var/www/html/strategist-ai/app/api/endpoints.py bootstrapr`,s KK9)EFB##')DT0UVV u @ 2bTR]a bwUhilmoiphqSr4stt B >ykA3OZ^ _wS5AB BBs0B30B3 B09A22 B0>-B++B00B3z /ingest/files.user_idcategoryfilesc Ktjd|dt| t||Dcgc]&}|j|j d{f(}}t ||||}|d}tj}ddt|d|d||d |jj|jjd |jd zd S7cc}w#t$rt$r>} tjd |d| dt!dddt#| dd} ~ wt$$r2} tjd|d| dt!ddddd} ~ wwxYww)Nz*Received ingest/files request for tenant: z , count: rbrasummaryrKz Ingested z files into . documents%H:%M:%SZ)rMrNrfrh ingested_date ingested_time created_atz'Validation/Extraction error for tenant rOTrPrRrSzFile ingestion failed: rLrTz,Unexpected error in ingest/files for tenant rWz(An error occurred during file ingestion.)rXrYlenrfilenamereadr/rFutcnowdate isoformattimestrftimer9rZrSrr[r\) rIrarbrcfuploadsresultrfnowr]r^s r_ ingest_filesrz9s KK?AAJJaffh/??!)WxQXY#oo"3u:,l9+QG , XXZ113 XXZ00<--/C/  !/?  y >ykB4P[_ `wUlmpqsmtluSv4wxx @ CI;bQRPST_c dwS}4~@sX%FC:#C5C3 C5"BC:2F3C55C::F 9E F-E>>FFz /ingest/text text_contentc Ktjd||js tdd t |t t j}tjd|tjtjz}|rY|jdj}tjdd|jjd d }|sd }nt||| }|d }t!||d}|D],}tjd|d|d|t#||.t%|} ||dd} |r|| d<tjdt'| d|d| D]} t)| ||} t+|| | |  t-|||} t/|d| t1j2}dd|d| |j5j7|j9j;d|j7dzdS#t<$rt>$r2}tjAd|d|d !td"d#d$d%d}~wwxYww)&Nz)Received ingest/text request for tenant: rRztext_content cannot be empty.rTz!^Title:\s*(.+?)\s*Content:\s*(.*)z[^\w\s-] _Documentraz.txttextzReplacing existing text z (sid: z ) in tenant )source source_idingestion_typerazCreated z chunks for text input ''rIrarKzIngested text content into rgrirj)rMrNrfrkrlrmz Error in ingest/text for tenant rOTrPrWrSz(An error occurred during text ingestion.rL)!rXrYstriprrr[uuiduuid4research IGNORECASEDOTALLgroupsubreplacer#rrr rnr!rr"rrFrqrrrsrtrur9r\rS)rIr{rar title_match display_titleunique_source_nameold_source_idssidchunksmetadatachunk embeddingrfryr^s r_ ingest_textr[sr  KK;I;GH    4STT1@# % ii DlTVTaTadfdmdmTmn '--a0668MFF;MBHHJRRSVX[\M * *<GTM -d30 ;MvV! 9C KK23E2FgcUR^_h^ij k (C 8 9L)0y\bc ")HY  hs6{m+CDVCWWXYZ FE&u 7SI y%H E F #<9gVy&'2oo4YKqA XXZ113 XXZ00<--/C/     @ 7 {"QCHSW XwS}4~@s)6I0G/H)(I0)I-;-I((I--I0z/ingest/websiteurlc Ktjd|d| tjddd4d{}|j |d{}|j dk\rQ|j dk7rBtj d |d |j tdd |j d | dddd{ t|tjd|d|t||t|dtj|d{}|d}|js(tjd|tdd| t!t#j$}||d|d|d|dd} |r|| d<|dD]} t'||| d} |j)dgD]m} | d} | d }t+| }|D]Q}d!|d"|}| j-}||d#<||d$<d| vr| d|d%<t/|||&}t1||||| d'z } Sotjd(| d)|d*t3|dd+|t5|||&}t7|d|t9j:}|j=j?| d,<|jAjCd-| d.<|j?d/z| d0<d1d2|d3t3|dd4|d5|| | d,| d.| d0d6S777O#1d{7swY`xYw#tj$r0}tjd|d|tdd| d}~wt$rwxYw7_#tD$rtF$r>}tjd7|d|d8tdd9d:t!|d; d}~wtH$r2}tjd<|d|d8td=d9d>d; d}~wwxYww)?Nz,Received ingest/website request for tenant: z, URL: g@FT)timeoutverifyfollow_redirectsrRizURL Validation failed for z with status code z+Website validation failed: Received status z from rTzURL Validation error for rOz0Website validation failed: Could not connect to zApplying replacement logic for z in tenant ) max_depth max_pagesrIrz#No content could be extracted from z5Failed to extract any content from the provided URL: website urls_visitedcolorsimages)rrrrr top_imagesrarpagesrz [Source: z] source_urlrrrr}zCreated total of z& chunks for website crawl starting at z (across z pages) with source_id rkrirlrjrmrKzIngested website content from z (visited z pages) into rg)rMrNrfrrkrlrmz/Website ingestion validation failed for tenant rPrSzWebsite ingestion failed: rLz.Unexpected website ingestion error for tenant rWz8An unexpected error occurred while crawling the website.)%rXrYhttpx AsyncClientheadrUwarningr RequestErrorrSrrr$rECRAWL_MAX_PAGESrr[rrrgetr copyr!rrnr"rrFrqrrrsrtrur9rZr\)rIrraclientrespr^ scrape_resultextracted_textr metadata_base visited_url total_chunkspage page_textpage_url page_chunksrreferenced_chunkchunk_metadatarrfryr]s r_ingest_websiters   KK>ykQTPUVW $$SQUV I IZ`S))D3&4+;+;s+B!;C5@RSWScScRdef#>ijnjzjzi{|BCFBG=HII  I I\P#  5cU+i[QR))S93C2QYQiQiu~ &v.##% LL>seD EC:opsot8uv v % "').9#H-'1   '.M) $)8 BK y)[ A B !%%gr2 "DV IE{H%Y/K$ "%.xjE7#C "/!3!3!5/7|,+3x(t#37>N<0*+;yZab "9.> >Z!  " "0  ' ~5[\_[``ijmn{}DoEkFjGG^_h^ij k#>YPWXy)W5oo),)=)=)? o&),)<)M;G/M> P  L,LL,L)L L)$L,,M8?+M**M88P ;M>>P9O  P-PPP z/ingest/sourcez7The original filename to delete from Azure Blob Storage) descriptionr source_namec :Ktjd|d| |r|d|}tj||t ||}dd|d|d|d|d S#t $r5}tj d |d |d |d tddddd}~wwxYww)zX Deletes all data associated with a specific source_id from the knowledge base. z&Received request to delete source_id 'z ' for tenant rrKzDeleted z chunks for source_id 'z' from z and Azure storage.)rMrN deleted_countzError deleting source_id for tenant rOTrPrWrSz'An error occurred during data deletion.rLrTN)rXrYr delete_filerr\rSr)rIrrazure_blob_namerr^s r_delete_ingested_sourcer s KK8 =QZP[\] !* 1[M:O  ' ' ? C4Y J !-0G {RYZcYddwx*   0 < {RTUVTWXcg hwS|4}~~s(B:AB B#0BBBz/summary/{type}typec 0Ktjd|d||dvr tdd t||\}}|s(tj d|d|td d ||||r|j j nd|r|jjd nd|r|j d Sdd S#t$r}|d}~wt$r5}tjd |d|d|dtddddd}~wwxYww)Nz+Received summary fetch request for tenant: z, type: )rcrrrRz8Invalid summary type. Use 'files', 'text', or 'website'.rTzNo summary found for z type z*No summary found for this type and tenant.ri)rIrrfrkrlrmz"Error fetching summary for tenant z, type rOTrPrWrSz Internal error fetching summary.rL) rXrYrrrrrrsrtrur\rS)rIrrfdther^s r_ get_summaryr's3 KK=i[QUPVWX //4noox(D9  NN29+VD6J KC8de e#68RWWY002d?ARWWY// ;t,.",,.   59     x 9)GD6QSTUSVWbf gwSu4vwwxsA-DBCDCD DC D0DDDcKtjd| t|d}d|t||dS#t$r2}tj d|d|dt d d d d  d}~wwxYww)Nz#Listing ingested files for tenant: filerK)rMrIcountrczError listing files for tenant rOTrPrWrSzError listing ingested files.rLrTrXrYrrnr\rSrrIsourcesr^s r_list_ingested_filesrAs KK5i[ABu' 6:#)c'l]dee u 6ykA3GRV WwSr4sttu%A68A6 A3-A..A33A6z/ingest/websitescKtjd| t|d}d|t||dS#t$r2}tj d|d|dt d d d d  d}~wwxYww)Nz&Listing ingested websites for tenant: rrK)rMrIrwebsitesz"Error listing websites for tenant rOTrPrWrSz Error listing ingested websites.rLrTrrs r_list_ingested_websitesrKs KK8 DEx' 9=#)c'l`ghh x 9)BqcJUY ZwSu4vwwxrcKtjd| t|d}d|t||dS#t$r2}tj d|d|dt d d d d  d}~wwxYww)Nz"Listing ingested text for tenant: rrK)rMrIr text_sourceszError listing text for tenant rOTrPrWrSzError listing ingested text.rLrTrrs r_list_ingested_textrUs KK4YK@At' 6:#)c'ldkll t 5i[1#FQU VwSq4rsstrz /documentscK t|}t||Dcgc]&}|j|jd{f(}}t ||||}ddt |ddd|S7/cc}w#t $rt$r5}tjd|d |d td d d|dd}~wt$r2}tjd|d |d tdd ddd}~wwxYww)zUpload documents under a category. Same indexing as /ingest/files, but the response carries each stored document's source_id, so the settings page can offer delete and re-categorise straight away without a second listing request. NrerKzStored rhz document(s).rLzExtraction error for tenant rOTrPrRrSzUpload failed: rTzError uploading documents for rWzCould not store the documents.) r7rrorpr/rnr9rZrXrSrr\) rIrbrarcrvrwrxr]r^s r_upload_documents_endpointr_s.c' 2 #9>?AAJJaffh/??!)WxQXY#$S )<%=$>mL !/?  Y 3I;bEPT UwBQRTQU@V5XY Y c 5i[1#FQU Vw@`5bc ccsWC;A8#A3A1 A3 'A80C;1A33A88C8 0B:: C8-C33C88C;c K t|}t||}|stddddddi|S#t$rt$r5}tj d|d |d |d td dddd}~wwxYww)z?Delete a document: its knowledge base rows and its stored file.rrSDocument not found.rLrTrMrKzError deleting document  for rOTrPrWzCould not delete the document.N)r7r,rr\rXrS)rIrremovedr^s r_delete_document_endpointrs c' 2 !)Y7C7DY9[\ \)/w//  c / {% {"QCP[_ `w@`5bc ccs%A=.3A=A:0A55A::A=cK t|}t||}dt||dS#t$r2}tj d|d|dt ddd d  d }~wwxYww) zHThe tenant's uploaded documents, for the Attach Documents settings card.rK)rMrrhzError listing documents for rOTrPrWrSzCould not list documents.rLrTN)r7r0rnr\rXrSr)rIrbrhr^s r_list_documents_endpointrs~^' 2 "9h7 #c)n9UU ^ 3I;bDt Tw@[5]^ ^^s$A(%*A( A%-A  A%%A(z/documents/categoryc"K t|}t|||stddddd||jxstdS#t$rt $r2}t jd|d |d td dd dd}~wwxYww)z+Re-categorise an already uploaded document.rrSrrLrTrK)rMrrbzError setting category for rOTrPrWzCould not update the category.N)r7r1rrr*r\rXrS)rIrrbr^s r_set_document_category_endpointrs c' 2 Iy(;C7DY9[\ \#)$NN,@0@B B  c 29+RsCd Sw@`5bc ccs)BAABB -BB  Bz/documents/download file_namecK|s|stdddd t|}|r t||n(t|t||xsij d}t ||}|stddddt |d S#t$rt$r6}tjd |xs|d |d tddddd}~wwxYww)a/Download a document. This is the URL handed to visitors in chat. It redirects to a storage link generated right now, so the link in a chat transcript never goes stale and the storage account never appears in a chat payload. Identify the document by `source_id`; `file_name` also works. rRrSz#source_id or file_name is required.rLrTrrri.)rUzError serving document rOTrPrWzCould not serve the document.N) rr7r.r-rr+r r\rXrS)rIrrdocumentrr^s r_document_download_endpointrs w@e5gh hb' 2 ;DM)Y7&y=I3V3\Z\(+K(8: y(+C7DY9[\ \ 55  b .y/EI.FbLW[ \w@_5ab bbs)CA,BCC 1CC  Cz/tools-settingscK t|}d|ttdS#t$r2}tj d|d|dt ddd d  d }~wwxYww) aLEvery capability switch for a tenant, plus the email copy. `features` carries all eight switches so the settings page can render them without knowing which ones the backend implements. `wired` names the subset that currently changes chat behaviour; the rest are stored preferences whose tools do not exist yet. rKrMrEwiredz!Error fetching tool settings for rOTrPrWrSzCould not load tool settings.rLrTN)r6sortedr4r\rXrSr)rIrEr^s r_get_tools_settings_endpointrs|b$Y/#$ .0 0 b 8 2aSITX Yw@_5ab bbs$A "A  A-AAA returnc,d}tDcic]}||vs||||}}d|vr|jd||dtDcic] }||vs|||}}tDcic]}|d|vr |||d}}|||fScc}wcc}wcc}w)zRead the flat form shape the settings page posts. Fields are named after the switches themselves (voice_mode=false), plus the three email strings. `enabled` is the old name for `send_email`. cTt|jjdvS)N)1trueyeson)r[rlower)raws r_as_boolz%_parse_settings_form..as_bools"3x~~%%'+EEEenabled send_email _provider)r3 setdefaultr2r5)formrnamefeaturesemail providerss r__parse_settings_formrs F7GWd$RV,gd4j))WHWDL'$y/*BC*= N$T4:  NE N=M/DV9%-ttfI.///I/ UI %%X O/s BB B B )BrequestcK|jjddjdr|jd{}t |t st dddd |jd xsdj}|jd xsi|jd xsi}}|jd xsi}|jd}nL|jd{}|jd xsdj}t|\}}}d}|st dddd |jD cic] \}} |  ||  }}} |jD cic] \}} |  ||  }}} |s|s|s|t dddd  tt|t||xsd|xsd|xsd|} d| ttdS77cc} }wcc} }w#t $r} t ddt#| d d} ~ wt$$r2} t&j)d|d| dt dddd d} ~ wwxYww)aUpdate capability switches and/or email copy. Only what is sent changes. Accepts either JSON: {"tenant_id": "...", "features": {"voice_mode": false}, "email": {"header": "..."}, "ticket_emails": [{"email": "...", "category": "..."}]} or form fields, which is what the settings page already posts: tenant_id=...&voice_mode=false&header=... Both are supported so moving off the old /email-settings URL is a one-line change on the caller rather than a rewrite of how it posts. `ticket_emails` is JSON-only: a list of objects has no natural flat-form encoding and nothing in this codebase's form-based caller needs it. z content-typer~zapplication/jsonNrRrSzBody must be an object.rLrTrIrrr ticket_emailsztenant_id is required.zNo settings supplied.)rrrr rKrzError saving tool settings for rOTrPrWzCould not save tool settings.)headersr startswithjson isinstancedictrrrritemsrr7r8rr4rZr[r\rXrS) rpayloadrIrrrr rkvrEr]r^s r_save_tools_settings_endpointrsk>2.99:LM &'4(C7D]9_` `[[-3::< !++j17RW9M9SQS%KK ,2  O4 \\^#XXk*0b779 %9$%?"% w@X5Z[ [ "*!1CAQ]1CHC#kkm =daq}QT =E = E) 8Mw@W5YZ Zb ,Y78 &i(:Jd,1MT090AT4AC$F=DYZZM'$D =" ]wSVWYSZ4[\\ b 6ykA3GRV Ww@_5ab bbs?IG B+I-G#.AI G%G%I1 G+<G+IAG1I#I% I1 I:H I-I  IIz/recommendations/eventrc PK|jdxsdj|jdxsdj}|jdxsdj|jdxsir|rjdstddd d  tvr/tddd d j t td   t jjfdd{}|jdsUtjd|djdd|jddjdd ddi|Sd tjjdd!}tj|dt!|d"d#|djdd|d$|S7#t$r?}tjd|djdd|dddddcYd}~Sd}~wwxYww)%aThe widget reports page activity; we decide whether to suggest anything. Answers HTTP 200 with `recommend: false` rather than an error whenever there is nothing to show, because a suggestion the visitor never asked for must never surface as a failure in their browser. rIr~ visitor_ideventrrrRrSz0tenant_id, visitor_id and page.url are required.rLrTz Unknown event. Expected one of: z, c0ttSN)r=r7)rrrIsr_z0recommendations_event_endpoint..UsF.y95$GrNz#Recommendation endpoint failed for z on rOTrPrKFno_match)rM recommendreasonrzNo recommendation for rz (title=title)rMrec_ productsz products for )rMrecommendation_id)rrrr<joinranyio to_threadrun_syncr\rXrSrYrrhexrn)rroutcomeexr"rrrIs @@@r_recommendations_event_endpointr*=s3[)/R668I++l+1r88:J [[ ! 'R . . 0E ;;v  $"D JdhhuoI5KL L L 9$))F /9)D U# G  ')H =wG G G  3"  H S NNHQ R R S n 7 {,ykY[\][^_jn owSk4lmmnscC B #AAAB C A B 'B?B B  B C 0CC  C c Ktjd|d| t||}dt||dS#t$r5}tj d|d|d|d t d d d d d}~wwxYww)Nz.Received request to fetch comments for ticket rJrK)rMrcommentsz#Error fetching comments for ticket rrOTrPrWrSzError retrieving comments.rLrT)rXrYrrnr\rSr)rFrIrXr^s r_fetch_ticket_commentsrYs KK@ =YbXcde r&y)<]   r :9+\R[Q\\^_`^abmq rwSo4pqqrs%A;:A; A80A33A88A;z/tickets/{ticket_id}/statusrMc *Ktjd|d|d|gd}||vrtddd|d  t|||}|std dd d  d d lm}t ||}ddd|d||xs||dd}|d|d|d{dd|||dS7#t$r"}tjd|Yd}~2d}~wwxYw#t$r5} tjd|d|d| dtddd d d} ~ wwxYww)!Nz,Received request to update status of ticket z to rJ)z In ProgressResolvedOngoingOpenrRrSzInvalid status. Must be one of rLrTrz"Ticket not found or update failed.rrKrMticket_status_updatedzTicket z status updated to )rFrMrOrQrRz1Failed to publish ticket status update to redis: rKzTicket status updated to )rMrNrF new_statusz!Error updating status for ticket rrOTrPrWzFailed to update ticket status.) rXrYrrrSrLrr\rrS) rFrIrMallowed_statusesrKrL ticket_datarrUr^s r_update_ticket_status_endpointrbs  KK>ykfXUbclbmnoE %%wUtvFuGTH5IJ Jw&y)VDC7W{8|} } Y >$Y :K 0$YK/B6(K#QY&'Q G  ')H =wG G G  26(;"   H Y NNNvhW X X Y w 8 B1O46B)GO   "  #V,  z A)BqcR]a bwSw4xyyzsP"B(AAAB(AB(A B%%A'' B%3-B  B%%B(z/chat/takeoverc `Kddlm}tjd|d|d|d| |||||}|s t dd  dd lm}|rd nd }|rd nd}d|rd nd |||dd} |d|d|| d{d|rd nd d|rdndd|||dd} |d|d| d{dd|rdndd||d S7A7#t$r"} tjd| Yd} ~ :d} ~ wwxYw#t$rt$r,} tjd!| d"#t d$d% d} ~ wwxYww)&zD Toggles human takeover for a specific conversation thread. r)set_takeover_statuszTakeover toggle: /z -> rrzConversation thread not found.rTrKtakeover_startedtakeover_endedzHuman agent has joined the chatz AI assistant has resumed controlrM)takeover_activerarOrQz:chat_Nz Takeover startedendedr)r.rz:conversation_listz,Could not publish takeover status to redis: rKzHuman takeover activated deactivated)rMrNrzTakeover toggle failed: TrPrWz'Failed to toggle human takeover status.) #app.services.integrations.firestorer|rXrYrrSrLr\rrS) rIr.rMrar|rKrL event_typemsgthread_payload list_payloadrUr^s r_ chat_takeoverr[sH KK#I;ay)DQR(_%iFGTC8XY Y T >/5+;KJ7=3CeC!/5+;K,2wG N  ')F9+ FW W W!/5+;K&Fy&Hi[Y&/FK L  ')4F GV V V  (M(RRWXaWbc%   X W T NNI&R S S T  _ /s3d C4]^^_sv(D.C-2B?9B;:,B?&B='B?+C-:D.;B?=B?? C*C% C-%C**C--D+?'D&&D++D.rc2|dtk7ry|jd}|sy|djd}|tn t|}|dkryt j t j|z }|t|k\ryt|t|dS) a}Whether this website source was read recently enough to skip. Only websites are throttled, and only when the config does not override the window -- a merchant who wants their site read on every press sets min_interval_hours to 0 without a deploy. Also never skips a source with zero product rows, regardless of last_synced_at -- that timestamp only means the source was crawled recently, not that the products from that crawl still exist. A cleanup that deletes product rows without also clearing this timestamp would otherwise leave a build that looks successful but serves an empty catalogue. kindFlast_synced_atconfigmin_interval_hoursr)hours external_ref) r@rCRAWL_MIN_INTERVAL_HOURSfloatrFryrHutcrGr>)rrI last_syncedrages r__crawled_recentlyrsf~#**-.K  8  !5 6E(- $5.runsGG "7!4 MIAv.vyA &*@+= ?@'QW-4wh/OP _  ))KFC&*@LGLM#SE)WQCtE7(+KL9 M:7## _  96.;Q&*,&*@4PR8K\K\]^^ _s!3B33 D .s c)nrsyncalready_runningrSjob_idqueuedrrM) rArrr$r%r&rDrBJobAlreadyRunningrrU) rIrrr6rsrrunningrs ` @r_catalog_sync_endpointrs)$G%I>):j)H1II 4OPP  C __--.DEEEC FC88 H 11%JF9  ! !C&'..5BC CCsWC BBAC,B-C2BBBCBC/C  CCz/catalog/enrichcK|r, tjjfdd{S tddfd d{}d |_ |ddS7*#t$r=}tj ddt dt|jd}~wwxYw7[#tj$r}t d d |jd d}~wwxYww)zRun attribute extraction over a tenant's catalog. Separate from /catalog/sync because it costs money and must be re-runnable without re-fetching a catalog. ctSrr:)rrIsr_rz)catalog_enrich_endpoint.. s- SX:YrNzEnrichment failed for %sTrPirTenrichc t||S)N)rr:)trrs r_rz)catalog_enrich_endpoint..s]1eh%Orrrrrrrr)r$r%r&r\rXrSrrrrDrBrrrU)rIrrr6r)rrs`` r_catalog_enrich_endpointrs  K112YZZ Z C x OQQH 11[ K LL3YL NCR8I8IJ J K Q  ! !C&'..5BC CCslC&AAACB&B$ B& CA B!$8BB!!C$B&&C9CCCr)NN)F)fastapirrrrrrr r r fastapi.responsesr typingr rapp.services.infra.databaserrrrrrrrrrrrrrrrapp.services.infra.storagerapp.services.infra.embeddingr r!r"r#app.services.content.websiter$app.services.chat.chatr%rr&r'app.services.chat.analyticsr(r)app.services.content.documentsr*r+r,r-r.r/r0r1 app.services.chat.tools_settingsr2r3r4r5r6r7r8app.services.infra.quotasr9app.services.enrichment.jobr;$app.services.pairing.recommendationsr<r=app.services.catalog.syncr>r?app.services.catalog.sourcesr@rAapp.services.infrarBapp.api.catalogrCrDapp.core.configrEr$asyncior loggingrrrrFrGrH getLoggerrrXrouterpostr[r`rzrrdeleterrrrrrrrrrrrtuplerrrr*r:r=r@rDrVrYrbintrhrkrzboolrrrlistrrrrrr_rs% ddd.! 7ee>4QB95EC@#+$   22   8 $ \ Bs B B_#Y!$Z"4j"3i @@ c]@sm@   @@B^#YS !$Z:@:@:@c]:@:@x #YCy!$ZrPrP rPc]rP rPh T/hi!2 xxCxx2Ouuu xCx xNttt\#Y"4j!$Z"3i ccsmcc]c   cc@|cccccc"L ^S ^HSM ^ ^ "##Y#YIcccc$c* !"FJ6:bbb03b#b> bbb&&%&* ;b;b ;b| %&9=c(T$(T'(TVW#Yc)_:$ZV "4jQQ QQ Q  Q  QsmQQBO6;Dk q# q q qO>GX]^bXc v v vRU v vJ q3 q q ,-#Y7m9 nnnn n.n@ +,AFs r3 r3 r- r *+#Ys)&w&w&w &w,&wP +0:E#JySycy y  !ysy"y" !E/YZD.NO$,JK %z%z%z%z %z"%zN #Y#Y9: 3_3_3_ 3_ 3_3_jKdKsKtKB(4((V_#Y $T e 22 2 2 228 #Yu+e 22 2 2 2 2r