a
(j%D @ s" d Z ddlmZ dZdgZddlmZ ddlmZm Z m
Z
m Z m Z m
Z
mZmZmZmZmZ ddlmZmZmZmZmZmZ ddlmZm Z dd l m Z m Z m Z m!Z! dd
l"m#Z# erdd l$m%Z% dd lm&Z& dd
l'm(Z(m)Z)m*Z* dZ+e e e,e,f e,e,gdf Z-G dd dee Z.G dd de Z/dS )zCUse the HTMLParser library to parse HTML files that aren't too bad. ) annotationsMITHTMLParserTreeBuilder)
HTMLParser) AnyCallablecastDictIterableListOptional
TYPE_CHECKINGTupleTypeUnion)
AttributeDictCDataComment DeclarationDoctypeProcessingInstruction)EntitySubstitution
UnicodeDammit)DetectsXMLParsedAsHTMLHTMLHTMLTreeBuilderSTRICTParserRejectedMarkup)
BeautifulSoup)NavigableString) _Encoding
_Encodings
_RawMarkupz html.parserNc @ s e Zd ZU dZded< dZded< edddd dd
d d Zd ed
< ded< ded< dddddZddddddZd3dddddd d Z d4dddd d d Z
ddd!d"d#Z ddd$d%d&Z ddd$d'd(Z
ddd!d)d*Zddd+d,d-Zddd!d.d/Zddd!d0d1Zd2S )5BeautifulSoupHTMLParserreplacestrREPLACEignoreIGNOREon_duplicate_attributer r z&Union[str, _DuplicateAttributeHandler])soupargsr+ kwargsc O s@ || _ || _|jj| _tj| g|R i | g | _| d S N)r, r+ builderattribute_dict_classr __init__ already_closed_empty_elementZ_initialize_xml_detector)selfr, r+ r- r. r5 =E:\soccer\ml_env\Lib\site-packages\bs4/builder/_htmlparser.pyr2 T s
z BeautifulSoupHTMLParser.__init__r+ z List[str]r3 r, None)messagereturnc C s t |d S r/ r )r4 r8 r5 r5 r6 erroro s z BeautifulSoupHTMLParser.errorz List[Tuple[str, Optional[str]]])tagattrsr9 c C s | j ||dd | | dS )zHandle an incoming empty-element tag.
html.parser only calls this method when the markup looks like
.
F)handle_empty_elementN)handle_starttag
handle_endtag)r4 r; r< r5 r5 r6 handle_startendtag s z*BeautifulSoupHTMLParser.handle_startendtagTbool)r; r< r= r9 c C s | }|D ]f\}}|du r d}||v rj| j}|| jkr:qr|d| jfv rR|||< qrtt|}|||| q |||< q | jjjr| \}} nd }} | jj
|dd||| d}
|
dur|
j r|r| j |dd | j
| | jdu r| | dS )zHandle an opening tag, e.g. ''
:param handle_empty_element: True if this tag is known to be
an empty-element tag (i.e. there is not expected to be any
closing tag).
N )
sourceline sourceposF)check_already_closed)r1 r+ r) r' r _DuplicateAttributeHandlerr, r0 Zstore_line_numbersgetposr> Zis_empty_elementr? r3 appendZ_root_tag_nameZ_root_tag_encountered) r4 r; r< r= Z attr_dictkeyvalueZon_duperC rD ZtagObjr5 r5 r6 r> s0
z'BeautifulSoupHTMLParser.handle_starttag)r; rE r9 c C s, |r || j v r | j | n | j| dS )zHandle a closing tag, e.g. ''
:param tag: A tag name.
:param check_already_closed: True if this tag is expected to
be the closing portion of an empty-element tag,
e.g. ''.
N)r3 remover, r? )r4 r; rE r5 r5 r6 r? s z%BeautifulSoupHTMLParser.handle_endtag)datar9 c C s | j | dS )z4Handle some textual data that shows up between tags.N)r, handle_datar4 rL r5 r5 r6 rM s z#BeautifulSoupHTMLParser.handle_data)namer9 c C sh | dr t|dd}n$| dr8t|dd}nt|}t|\}}|rZd| j_| | dS )zHandle a numeric character reference by converting it to the
corresponding Unicode character and treating it as textual
data.
:param name: Character number, possibly in hexadecimal.
x XTN)
startswithintlstripr Znumeric_character_referencer, contains_replacement_charactersrM )r4 rO Z real_namerL Zreplacement_addedr5 r5 r6 handle_charref s
z&BeautifulSoupHTMLParser.handle_charrefc C s0 t j|}|dur|}nd| }| | dS )zHandle a named entity reference by converting it to the
corresponding Unicode character(s) and treating it as textual
data.
:param name: Name of the entity reference.
Nz&%s)r ZHTML_ENTITY_TO_CHARACTERgetrM )r4 rO characterrL r5 r5 r6 handle_entityref s
z(BeautifulSoupHTMLParser.handle_entityrefc C s& | j | j | | j t dS )zOHandle an HTML comment.
:param data: The text of the comment.
N)r, endDatarM r rN r5 r5 r6 handle_comment
s
z&BeautifulSoupHTMLParser.handle_comment)declr9 c C s6 | j |tdd }| j | | j t dS )zYHandle a DOCTYPE declaration.
:param data: The text of the declaration.
zDOCTYPE N)r, r[ lenrM r )r4 r] r5 r5 r6 handle_decl s
z#BeautifulSoupHTMLParser.handle_declc C sN | dr$t}|tdd }nt}| j | j| | j| dS )z{Handle a declaration of unknown type -- probably a CDATA block.
:param data: The text of the declaration.
zCDATA[N)upperrS r r^ r r, r[ rM )r4 rL clsr5 r5 r6 unknown_decl s
z$BeautifulSoupHTMLParser.unknown_declc C s0 | j | j | | | | j t dS )z\Handle a processing instruction.
:param data: The text of the instruction.
N)r, r[ rM Z_document_might_be_xmlr rN r5 r5 r6 handle_pi/ s
z!BeautifulSoupHTMLParser.handle_piN)T)T)__name__
__module__ __qualname__r' __annotations__r) r2 r: r@ r> r? rM rW rZ r\ r_ rb rc r5 r5 r5 r6 r$ = s&
>
r$ c s e Zd ZU dZdZded< dZded< eZded< ee e
gZ d ed
< d ed < dZ ded
< d!dddd fdd
Z
d"ddddddddZefdd d d d d Z ZS )#r zA Beautiful soup `bs4.builder.TreeBuilder` that uses the
:py:class:`html.parser.HTMLParser` parser, found in the Python
standard library.
FrA is_xmlT picklabler& NAMEz
Iterable[str]featuresz$Tuple[Iterable[Any], Dict[str, Any]] parser_argsTRACKS_LINE_NUMBERSNzOptional[Iterable[Any]]zOptional[Dict[str, Any]]r )rl
parser_kwargsr. c sp t }dD ] }||v r
||}|||< q
tt| jf i | |pFg }|pNi }|| d|d< ||f| _dS )a Constructor.
:param parser_args: Positional arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param parser_kwargs: Keyword arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param kwargs: Keyword arguments for the superclass constructor.
r* Fconvert_charrefsN)dictpopsuperr r2 updaterl )r4 rl rn r. Zextra_parser_kwargsargrJ __class__r5 r6 r2 K s
z HTMLParserTreeBuilder.__init__r# zOptional[_Encoding]zOptional[_Encodings]zDIterable[Tuple[str, Optional[_Encoding], Optional[_Encoding], bool]])markupuser_specified_encodingdocument_declared_encodingexclude_encodingsr9 c c s t |tr |dddfV dS g }|r.|| g }|r@|| t|||d|d}|jdu rftdn|j|j|j|jfV dS )a2 Run any preliminary steps necessary to make incoming markup
acceptable to the parser.
:param markup: Some markup -- probably a bytestring.
:param user_specified_encoding: The user asked to try this encoding.
:param document_declared_encoding: The markup itself claims to be
in this encoding.
:param exclude_encodings: The user asked _not_ to try any of
these encodings.
:yield: A series of 4-tuples: (markup, encoding, declared encoding,
has undergone character replacement)
Each 4-tuple represents a strategy for parsing the document.
This TreeBuilder uses Unicode, Dammit to convert the markup
into Unicode, so the ``markup`` element of the tuple will
always be a string.
NFT)known_definite_encodingsuser_encodingsZis_htmlrz zPCould not convert input to Unicode, and html.parser will not accept bytestrings.)
isinstancer& rH r Zunicode_markupr Zoriginal_encodingZdeclared_html_encodingrV )r4 rw rx ry rz r{ r| Zdammitr5 r5 r6 prepare_markupi s2
z$HTMLParserTreeBuilder.prepare_markupz type[BeautifulSoupHTMLParser]r7 )rw
_parser_classr9 c
C s | j \}}t|tsJ | jdus&J || jg|R i |}z|| | W n, ty } zt|W Y d}~n
d}~0 0 g |_dS )z
:param markup: The markup to feed into the parser.
:param _parser_class: An HTMLParser subclass to use. This is only intended for use in unit tests.
N) rl r} r& r, feedcloseAssertionErrorr r3 )r4 rw r r- r. parserer5 r5 r6 r s
zHTMLParserTreeBuilder.feed)NN)NNN)rd re rf __doc__rh rg ri
HTMLPARSERrj r r rk rm r2 r~ r$ r
__classcell__r5 r5 ru r6 r : s
! H)0r
__future__r __license____all__ html.parserr typingr r r r r
r r r
r r r Z bs4.elementr r r r r r Z
bs4.dammitr r Z bs4.builderr r r r Zbs4.exceptionsr Zbs4r r Z bs4._typingr! r" r# r r& rF r$ r r5 r5 r5 r6 s& 4 ~