a bg1@spddlZddlZddlZddlmZmZddlZddlmZddl m Z e dZ e dZ Gddde ZdS) N)ListTuple)Document) BaseLoaderzBV\w+zav[0-9]+c@sTeZdZdZd eeeeedddZeedddZee ee fd d d Z d S)BiliBiliLoaderz9 Load fetching transcripts from BiliBili videos. ) video_urlssessdatabili_jctbuvid3cCsZ||_d|_zddlm}Wnty6tdYn0|rV|rV|rV|j|||d|_dS)a Initialize the loader with BiliBili video URLs and authentication cookies. if no authentication cookies are provided, the loader can't get transcripts and will only fetch videos info. Args: video_urls (List[str]): List of BiliBili video URLs. sessdata (str): SESSDATA cookie value for authentication. bili_jct (str): BILI_JCT cookie value for authentication. buvid3 (str): BUVI3 cookie value for authentication. Nr)videoTrequests package not found, please install it with `pip install bilibili-api-python`)r r r )r credential bilibili_apir ImportErrorZ Credential)selfrr r r r r{/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/document_loaders/bilibili.py__init__s   zBiliBiliLoader.__init__)returncCs8g}|jD](}||\}}t||d}||q |S)z Load and return a list of documents containing video transcripts. Returns: List[Document]: List of Document objects transcripts and metadata. )Z page_contentmetadata)r_get_bilibili_subs_and_inforappend)rresultsurlZ transcript video_infodocrrrload5s    zBiliBiliLoader.load)rrcCst|}zddlm}m}Wnty8tdYn0|rT|j||jd}n>t |}|r|jt |dd|jd}nt d||| }| d |i|jsd |fS|||d }|d g} | r| dd d } | dsd| } t| } | jdkrft| jdg} ddd| D} d|dd|dd| }||fStd|d| jntd|dd |fS)zU Retrieve video information and transcript for a given BiliBili URL. r)syncr r )bvidrN)aidrz(Unable to find a valid video ID in URL: rrZcidZ subtitlesZ subtitle_urlhttpzhttps:body cSsg|] }|dqS)contentr).0crrr kz>BiliBiliLoader._get_bilibili_subs_and_info..z Video Title: titlez, description: descz Transcript: zFailed to fetch subtitles for z. HTTP Status Code: zNo subtitles found for video: z. Returning empty transcript.) BV_PATTERNsearchrrr rZVideogroupr AV_PATTERNint ValueErrorget_infoupdateZ get_subtitleget startswithrequests status_codejsonloadsr&joinwarningswarn)rrrrr vr!rsubZsub_listZsub_urlresponseZraw_sub_titlesZraw_transcriptZraw_transcript_with_meta_inforrrrDsV    "       z*BiliBiliLoader._get_bilibili_subs_and_infoN)rrr) __name__ __module__ __qualname____doc__rstrrrrrdictrrrrrrs r)r9rer<typingrrr7Zlangchain_core.documentsrZ)langchain_community.document_loaders.basercompiler-r0rrrrrs