qujB ddlmZddlZddlZddlmZmZddlZddlm Z m Z m Z m Z ddl mZddlZej eZdedefdZd ed edefd Zdd ed edede ede ee ff dZy)) get_browserN)urljoinurlparse)ListDictSetAny) BeautifulSouphtmlreturnc g}t|d}|jdddiD]}|jxs|jxsd} t j |}t|tr|n|g}|D]_}t|tst|jdtr|jd |dDO|j|a|S#tj tf$rtjdYwxYw) zEvery JSON-LD object on a page, with @graph wrappers flattened. Must run before the crawler decomposes