HOME


Mini Shell 1.0
Redirecting to https://devs.lapieza.net/iniciar-sesion Redirecting to https://devs.lapieza.net/iniciar-sesion.
DIR: /proc/self/root/usr/lib/python3/dist-packages/charset_normalizer/__pycache__/
Upload File :
Current File : //proc/self/root/usr/lib/python3/dist-packages/charset_normalizer/__pycache__/cd.cpython-311.pyc
�

�c1�
�~�ddlZddlmZddlmZddlmZddlmZm	Z	m
Z
mZmZddl
mZddlmZmZmZmZdd	lmZdd
lmZddlmZmZmZmZmZded
e
efd�Zded
e
efd�Z e��ded
e
efd���Z!e��ded
e
efd���Z"ee���ded
ee#e#ffd���Z$	d&de
ede#d
e
efd�Z%dede
ed
e&fd�Z'ded
e
efd�Z(de
ed
efd�Z)ded
efd �Z*ed!���	d'ded#e&d$eed
efd%���Z+dS)(�N)�IncrementalDecoder)�Counter)�	lru_cache)r�Dict�List�Optional�Tuple�)�FREQUENCIES)�KO_NAMES�LANGUAGE_SUPPORTED_COUNT�TOO_SMALL_SEQUENCE�ZH_NAMES)� is_suspiciously_successive_range)�CoherenceMatches)�is_accentuated�is_latin�is_multi_byte_encoding�is_unicode_range_secondary�
unicode_range�	iana_name�returnc�����t|��rtd���tjd�|����j}|d���}i�d�t
dd��D]h}|�t|g����}|rAt|��}|��9t|��d	ur|�vrd�|<�|xxd
z
cc<�d
z
��it��fd��D����S)zF
    Return associated unicode ranges in a single byte code page.
    z.Function not supported on multi-byte code pagezencodings.{}�ignore)�errorsr�@�NFr
c�2��g|]}�|�zdk�|��S)g333333�?�)�.0�character_range�character_count�seen_rangess  ���7/usr/lib/python3/dist-packages/charset_normalizer/cd.py�
<listcomp>z*encoding_unicode_range.<locals>.<listcomp>3s8���	
�	
�	
���?�+�o�=��E�E�
�E�E�E�)r�IOError�	importlib�
import_module�formatr�range�decode�bytesrr�sorted)r�decoder�p�i�chunkr!r"r#s      @@r$�encoding_unicode_ranger3sI�����i�(�(�H��F�G�G�G��%����i�(�(�����$�G�8�4�4�4�A�"$�K��O�
�4��
�
�
%�
%���X�X�e�Q�C�j�j�)�)���
	%�-:�5�-A�-A�O��&��)�/�:�:�e�C�C�"�+�5�5�34�K��0��O�,�,�,��1�,�,�,��1�$����	
�	
�	
�	
�	
�#.�	
�	
�	
���r&�
primary_rangec��g}tj��D]4\}}|D],}t|��|kr|�|��n�-�5|S)z>
    Return inferred languages used with a unicode range.
    )r�itemsr�append)r4�	languages�language�
characters�	characters     r$�unicode_range_languagesr<;so���I� +� 1� 3� 3�����*�#�	�	�I��Y�'�'�=�8�8�� � ��*�*�*���9���r&c�f�t|��}d}|D]
}d|vr|}n�|�dgSt|��S)z�
    Single-byte encoding language association. Some code page are heavily linked to particular language(s).
    This function does the correspondence.
    N�Latin�Latin Based)r3r<)r�unicode_rangesr4�specified_ranges    r$�encoding_languagesrBJs]��!7�y� A� A�N�#'�M�)�����/�)�)�+�M��E�*������"�=�1�1�1r&c��|�d��s0|�d��s|�d��s|dkrdgS|�d��s	|tvrdgS|�d��s	|tvrd	gSgS)
z�
    Multi-byte encoding language association. Some code page are heavily linked to particular language(s).
    This function does the correspondence.
    �shift_�
iso2022_jp�euc_j�cp932�Japanese�gb�Chinese�
iso2022_kr�Korean)�
startswithrr)rs r$�mb_encoding_languagesrN^s���	���X�&�&������-�-������(�(�������|�����D�!�!��Y�(�%:�%:��{�����L�)�)��Y�(�-B�-B��z��
�Ir&)�maxsizer9c��d}d}t|D]*}|st|��rd}|rt|��durd}�+||fS)zg
    Determine main aspects from a supported language if it contains accents and if is pure Latin.
    FT)rrr)r9�target_have_accents�target_pure_latinr;s    r$�get_target_featuresrSssn��
!&��"�� ��*�&�&�	�"�	'�~�i�'@�'@�	'�"&���	&��)�!4�!4��!=�!=� %���� 1�1�1r&Fr:�ignore_non_latinc���g}td��D����}tj��D]q\}}t|��\}}|r|dur�|dur|r�%t	|��}t	�fd�|D����}	|	|z}
|
dkr|�||
f���rt
|d�d���}d�|D��S)	zE
    Return associated languages associated to given characters.
    c3�4K�|]}t|��V��dS�N)r)r r;s  r$�	<genexpr>z%alphabet_languages.<locals>.<genexpr>�s*����T�T�I�n�Y�7�7�T�T�T�T�T�Tr&Fc���g|]}|�v�|��	Srr)r �cr:s  �r$r%z&alphabet_languages.<locals>.<listcomp>�s���?�?�?�1�q�J���Q���r&g�������?c��|dS�Nr
r��xs r$�<lambda>z$alphabet_languages.<locals>.<lambda>�s
���!��r&T��key�reversec��g|]
}|d��S)rr)r �compatible_languages  r$r%z&alphabet_languages.<locals>.<listcomp>�s��H�H�H�':���"�H�H�Hr&)�anyrr6rS�lenr7r.)r:rTr8�source_have_accentsr9�language_charactersrQrRr"�character_match_count�ratios`          r$�alphabet_languagesrk�s���*,�I��T�T��T�T�T�T�T��)4�):�)<�)<�0�0�%��%�1D�X�1N�1N�.��.��	� 1�U� :� :���%�'�'�,?�'��"�#6�7�7��%(�?�?�?�?�+�?�?�?�&
�&
��-��>���C�<�<����h��.�/�/�/���y�n�n�d�C�C�C�I�H�H�i�H�H�H�Hr&�ordered_charactersc�d�|tvr"td�|�����d}tt|��}t	|��}t	t|��}|dk}t|t
d|����D�]�\}}||vr�t|�|��}	||z}
t||
z��}|durt||	z
��dkr�]|durt||	z
��|dzkr|dz
}��t|d|	�}t||	d	�}
|d|�}||d	�}t	t|��t|��z��}t	t|��t|
��z��}t	|��dkr
|dkr|dz
}��6t	|
��dkr
|dkr|dz
}��V|t	|��zd
ks|t	|
��zd
kr|dz
}������|t	|��zS)aN
    Determine if a ordered characters list (by occurrence from most appearance to rarest) match a particular language.
    The result is a ratio between 0. (absolutely no correspondence) and 1. (near perfect fit).
    Beware that is function is not strict on the match in order to ease the detection. (Meaning close match is 1.)
    z{} not availabler�F�T�r
Ng�������?)
r�
ValueErrorr*�setrf�zipr+�index�int�abs)r9rl�character_approved_count�FREQUENCIES_language_set�ordered_characters_count� target_language_characters_count�large_alphabetr;�character_rank�character_rank_in_language�expected_projection_ratio�character_rank_projection�characters_before_source�characters_after_source�characters_before�characters_after�before_match_count�after_match_counts                  r$�characters_popularity_comparer��s����{�"�"��+�2�2�8�<�<�=�=�=�$%��"�;�x�#8�9�9��$'�(:�$;�$;��,/��H�0E�,F�,F�$�;�b�@�N�%(��E�!�%=�>�>�&�&�8�8�!�	�>��4�4�4��*5�h�*?�*E�*E�i�*P�*P�"�,�/G�G�	"�*-�^�>W�-W�)X�)X�!�
�e�#�#��-�0J�J�K�K�a�O�O��
�d�"�"��-�0J�J�K�K�.��2�3�3�
%��)�$��.9�(�.C�
�(�(�/
� �.9��-B�&�'�'�.
��(:�!�N�:J�'K��&8����&I��"%��!�"�"�S�)A�%B�%B�B�#
�#
��"%�� �!�!�C�(?�$@�$@�@�"
�"
���'�(�(�A�-�-�2D��2I�2I�$��)�$���&�'�'�1�,�,�1B�a�1G�1G�$��)�$��
��%=�!>�!>�>�#�E�E� �3�'>�#?�#?�?�3�F�F�$��)�$��G�
$�c�*<�&=�&=�=�=r&�decoded_sequencec�b�i}|D]�}|���dur�t|��}|��+d}|D]}t||��dur|}n�|�|}||vr|���||<�h||xx|���z
cc<��t	|�����S)a
    Given a decoded text sequence, return a list of str. Unicode range / alphabet separation.
    Ex. a text containing English/Latin with a bit a Hebrew will return two items in the resulting list;
    One containing the latin letters and the other hebrew.
    FN)�isalpharr�lower�list�values)r��layersr;r!�layer_target_range�discovered_ranges      r$�alpha_unicode_splitr��s��� �F�%�8�8�	������%�'�'��)6�y�)A�)A���"��,0�� &�	�	��0�1A�?�S�S����&6�"���	��%�!0���V�+�+�)2���):�):�F�%�&���!�"�"�"�i�o�o�&7�&7�7�"�"�"�"���
�
��� � � r&�resultsc���i�|D]2}|D]-}|\}}|�vr|g�|<��|�|���.�3�fd��D��}t|d�d���S)z�
    This function merge results previously given by the function coherence_ratio.
    The return type is the same as coherence_ratio.
    c
���g|]=}|tt�|��t�|��zd��f��>S)ro)�round�sumrf)r r9�per_language_ratioss  �r$r%z*merge_coherence_ratios.<locals>.<listcomp>-sd���	
�	
�	
�
�

���'��1�2�2�S�9L�X�9V�5W�5W�W��
�
�	
�	
�	
�	
r&c��|dSr\rr]s r$r_z(merge_coherence_ratios.<locals>.<lambda>8s
��q��t�r&Tr`)r7r.)r��result�
sub_resultr9rj�merger�s      @r$�merge_coherence_ratiosr�s����
35���8�8�� �	8�	8�J�(�O�H�e��2�2�2�16��#�H�-����)�0�0��7�7�7�7�	8�	
�	
�	
�	
�,�	
�	
�	
�E��%�^�^�T�:�:�:�:r&c�H��t���|D]A}|\}}|�dd��}|�vrg�|<�|�|���Bt�fd��D����r3g}�D],}|�|t	�|��f���-|S|S)u�
    We shall NOT return "English—" in CoherenceMatches because it is an alternative
    of "English". This function only keeps the best match and remove the em-dash in it.
    u—�c3�J�K�|]}t�|��dkV��dS)r
N)rf)r �e�
index_resultss  �r$rXz/filter_alt_coherence_matches.<locals>.<genexpr>Ks5�����
<�
<��3�}�Q�� � �1�$�
<�
<�
<�
<�
<�
<r&)�dict�replacer7re�max)r�r�r9rj�
no_em_name�filtered_resultsr�s      @r$�filter_alt_coherence_matchesr�;s����
-1�F�F�M��0�0�� ���%�"�*�*�5�"�5�5�
��]�*�*�(*�M�*�%��j�!�(�(��/�/�/�/�
�
<�
<�
<�
<�m�
<�
<�
<�<�<� �-/��%�	N�	N�H��#�#�X�s�=��3J�/K�/K�$L�M�M�M�M����Nr&i皙�����?�	threshold�lg_inclusionc	�J�g}d}d}|�|�d��ng}d|vrd}|�d��t|��D]�}t|��}|���}	td�|	D����}
|
tkr�Jd�|	D��}|pt||��D]Q}t||��}
|
|kr�|
d	kr|d
z
}|�	|t|
d��f��|dkrn�R��tt|��d
�d���S)z�
    Detect ANY language that can be identified in given sequence. The sequence will be analysed by layers.
    A layer = Character extraction by alphabets/ranges.
    FrN�,r?Tc3� K�|]	\}}|V��
dSrWr�r rZ�os   r$rXz"coherence_ratio.<locals>.<genexpr>ms&����"=�"=���A�1�"=�"=�"=�"=�"=�"=r&c��g|]\}}|��Srrr�s   r$r%z#coherence_ratio.<locals>.<listcomp>rs��/J�/J�/J�d�a���/J�/J�/Jr&g�������?r
rorpc��|dSr\rr]s r$r_z!coherence_ratio.<locals>.<lambda>�s
��Q�q�T�r&r`)
�split�remover�r�most_commonr�rrkr�r7r�r.r�)r�r�r�r�rT�sufficient_match_count�lg_inclusion_list�layer�sequence_frequenciesr�r"�popular_character_orderedr9rjs              r$�coherence_ratior�Vs���(*�G�"��"#��3?�3K��*�*�3�/�/�/�QS���)�)�)���� � ��/�/�/�$�%5�6�6����18�����*�6�6�8�8��"�"=�"=��"=�"=�"=�=�=���0�0�0��/J�/J�k�/J�/J�/J�!�)�
�-?�%�'7�.
�.
�	�	�H�9��3���E��y� � ���#���&�!�+�&��N�N�H�e�E�1�o�o�6�7�7�7�%��*�*���+���$�W�-�-�>�>�4����r&)F)r�N),r(�codecsr�collectionsr�	functoolsr�typing�TypeCounterrrrr	�assetsr�constantrr
rr�mdr�modelsr�utilsrrrrr�strr3r<rBrN�boolrSrk�floatr�r�r�r�r�rr&r$�<module>r�sk������%�%�%�%�%�%�������������F�F�F�F�F�F�F�F�F�F�F�F�F�F�������V�V�V�V�V�V�V�V�V�V�V�V�0�0�0�0�0�0�$�$�$�$�$�$���������������$�c�$�d�3�i�$�$�$�$�N�3��4��9���������2�#�2�$�s�)�2�2�2�
��2�&�����S��T�#�Y����
���(��+�,�,�,�
2�#�
2�%��d�
�*;�
2�
2�
2�-�,�
2�"5:�!I�!I��S�	�!I�-1�!I�	�#�Y�!I�!I�!I�!I�HM>��M>�'+�C�y�M>�
�M>�M>�M>�M>�`$!�#�$!�$�s�)�$!�$!�$!�$!�N;�D�)9�$:�;�?O�;�;�;�;�8�*:��?O�����6��4����QU�0�0��0�&+�0�AI�#��0��0�0�0���0�0�0r&