`L iŕ dZdZdZdZddlZddlZddlmZddlmZm Z m Z m Z m Z m Z mZgdZd Zd Zd Zd Zd Zej*d ej,Zej*dej,Zej*dej,Zej*dZej*dej,Zej*dej,Ze e Zee dfZerddlm Z Gdde Z!ne e"e fZ!dZ#e#\Z$Z%ddddddddd d Z&e'd!Dcic]}e(|d"|zc}Z)e)jUe&jWDcic]\}}|| c}}de)d#<e&jUe'd$Dcic]}d%|ze(|c}d&Z,d'Z-d(Z.dZ/d)Z0d*Z1d+Z2d,Z3e/e0e1e2e3gZ4Gd-d.e5Z6Gd/d0e6Z7Gd1d2e6Z8Gd3d4e6Z9Gd5d6e6Z:Gd7d8e6Z;Gd9d:e6Z<Gd;de6Z>Gd?d@e6Z?GdAdBe6Z@GdCdDe6ZAdEZBdFZCGdGdHZDGdIdJZEGdKdLZFGdMdNZGGdOdPeGeFZHGdQdRZIGdSdTZJGdUdVeGeJZKdWZLdXZMGdYdZZNGd[d\ZOd]e/fd^ZPd]e/fd_ZQd`ZRdaZSycc}wcc}}wcc}w)ba The liac-arff module implements functions to read and write ARFF files in Python. It was created in the Connectionist Artificial Intelligence Laboratory (LIAC), which takes place at the Federal University of Rio Grande do Sul (UFRGS), in Brazil. ARFF (Attribute-Relation File Format) is an file format specially created for describe datasets which are commonly used for machine learning experiments and software. This file format was created to be used in Weka, the best representative software for machine learning automated experiments. An ARFF file can be divided into two sections: header and data. The Header describes the metadata of the dataset, including a general description of the dataset, its name and its attributes. The source below is an example of a header section in a XOR dataset:: % % XOR Dataset % % Created by Renato Pereira % rppereira@inf.ufrgs.br % http://inf.ufrgs.br/~rppereira % % @RELATION XOR @ATTRIBUTE input1 REAL @ATTRIBUTE input2 REAL @ATTRIBUTE y REAL The Data section of an ARFF file describes the observations of the dataset, in the case of XOR dataset:: @DATA 0.0,0.0,0.0 0.0,1.0,1.0 1.0,0.0,1.0 1.0,1.0,0.0 % % % Notice that several lines are starting with an ``%`` symbol, denoting a comment, thus, lines with ``%`` at the beginning will be ignored, except by the description part at the beginning of the file. The declarations ``@RELATION``, ``@ATTRIBUTE``, and ``@DATA`` are all case insensitive and obligatory. For more information and details about the ARFF file description, consult http://www.cs.waikato.ac.nz/~ml/weka/arff.html ARFF Files in Python ~~~~~~~~~~~~~~~~~~~~ This module uses built-ins python objects to represent a deserialized ARFF file. A dictionary is used as the container of the data and metadata of ARFF, and have the following keys: - **description**: (OPTIONAL) a string with the description of the dataset. - **relation**: (OBLIGATORY) a string with the name of the dataset. - **attributes**: (OBLIGATORY) a list of attributes with the following template:: (attribute_name, attribute_type) the attribute_name is a string, and attribute_type must be an string or a list of strings. - **data**: (OBLIGATORY) a list of data instances. Each data instance must be a list with values, depending on the attributes. The above keys must follow the case which were described, i.e., the keys are case sensitive. The attribute type ``attribute_type`` must be one of these strings (they are not case sensitive): ``NUMERIC``, ``INTEGER``, ``REAL`` or ``STRING``. For nominal attributes, the ``atribute_type`` must be a list of strings. In this format, the XOR dataset presented above can be represented as a python object as:: xor_dataset = { 'description': 'XOR Dataset', 'relation': 'XOR', 'attributes': [ ('input1', 'REAL'), ('input2', 'REAL'), ('y', 'REAL'), ], 'data': [ [0.0, 0.0, 0.0], [0.0, 1.0, 1.0], [1.0, 0.0, 1.0], [1.0, 1.0, 0.0] ] } Features ~~~~~~~~ This module provides several features, including: - Read and write ARFF files using python built-in structures, such dictionaries and lists; - Supports `scipy.sparse.coo `_ and lists of dictionaries as used by SVMLight - Supports the following attribute types: NUMERIC, REAL, INTEGER, STRING, and NOMINAL; - Has an interface similar to other built-in modules such as ``json``, or ``zipfile``; - Supports read and write the descriptions of files; - Supports missing values and names with spaces; - Supports unicode values and names; - Fully compatible with Python 2.7+, Python 3.5+, pypy and pypy3; - Under `MIT License `_ z7Renato de Pontes Pereira, Matthias Feurer, Joel NothmanzTrenato.ppontes@gmail.com, feurerm@informatik.uni-freiburg.de, joel.nothman@gmail.comz2.4.0N) TYPE_CHECKING)OptionalListDictAnyIteratorUnionTupleNUMERICREALINTEGERSTRING%z @RELATIONz @ATTRIBUTEz@DATAz^([^\{\}%,\s]*|\".*\"|\'.*\')$z%^(\".*\"|\'.*\'|[^\{\}%,\s]*)\s+(.+)$z["\'\\\s%,\000-\031]z(?=["\'\\%])|[\n\r\t\000-\031]z^\s*\{.*\}\s*$z["'{}\s].) TypedDictc@eZdZUeed<eed<eed<eeefed<y)ArffContainerType descriptionrelation attributesdataN) __name__ __module__ __qualname__str__annotations__rr ArffDenseDataTypeArffSparseDataType]/mnt/ssd/data/python-lab/Trading/venv/lib/python3.12/site-packages/sklearn/externals/_arff.pyrrs% %'99::r rcd}d|d|jddd}tjdj|}tjd d |iz}||fS) Na " # open quote followed by zero or more of: (?: (?""3qua=!!  D>BC C Ds A!!A9cZ|dddvrtjdt|ddS|dvry|S)Nr9)r#r$z\\([0-9]{1,3}|u[0-9a-f]{4}|.)?r5)r'subrF)vs r!_unquoterMs?!u vv68L"g  ir c@tj|s3ttj|gDcgc] }|dvrdn| c}St t jd|z\}}t|s|Dcgc] }t|c}Stj|r< tj|Dcic]\}}t|t|c}}St j|D]/}|j!dst#d|j!zt#dcc}wcc}wcc}}w#t$rStj|D]/}|j!drt#d|j!zt#dwxYw)z-(INTERNAL) Split a line into a list of valuesrIN,r9zError parsing %rzUnknown parsing errorr8)_RE_NONTRIVIAL_DATAsearchnextcsvreaderzip_RE_DENSE_VALUESfindallanyrM_RE_SPARSE_LINErD_RE_SPARSE_KEY_VALUESrCrAfinditerr= BadLayout)rEvalueserrorsrLkrDs r! _parse_valuesr`s  % %a (cjj!o.0YA-0 0*2237;rurange_decode_values)ristream conversorsrowr]rs r! decode_rowszDenseGeneratorData.decode_rowss :C"3'F&$'c&kS_<',, J02aqF{&)922v;#j/1',,%%fj9 9 :2sAB0B+/AB0c t||Dcgc]\}}|dn||}}}|Scc}}w#t$r"}dt|vr tYd}~|Sd}~wwxYwNzfloat: )rUrArr)r]r conversorr{excs r!rz!DenseGeneratorData._decode_valuess| *Z02*)U$md51AA2F2  2 *CH$'))%  *s 2,22 AAAc #<Kd}|D]}t|t|k7r#td|t|t|fzg}|D]6}| |dk(s||k7rd}ntt|}|j |8|dz }dj |yw)aZ(INTERNAL) Encodes a line of data. Data instances follow the csv format, i.e, attribute values are delimited by commas. After converted from csv. :param data: a list of values. :param attributes: a list of attributes. Used to check if data is valid. :return: a string with the encoded data line. r*Instance %d has %d attributes, expected %dNr5rJr9rO)r>rrrappendjoin)rirr current_rowinstnew_datar{rEs r! encode_datazDenseGeneratorData.encode_datas  %D4yC O+@!3t9c*o>? H #=ERK5E>A%c%j1A"  # 1 K((8$ $! %sBBN)rrrrqr staticmethodrrrr r!rrs%;: %r rc"eZdZdZfdZxZS)_DataListMixinz>Mixin to return a list from decode_rows instead of a generatorc6tt| ||Srl)listryr)rirrr|s r!rz_DataListMixin.decode_rowssEG' ;<) rirrrrowscolsrrr]row_colskeyr{rs r!rzCOOData.decode_rowss(r2Dd' "FAs"3'Ffd+k!"F6<<>$:; Hf ),/&,AC(c5$)=%ojoe6LLCC KK  KKc&k) * KK !) ",T4C C(+-- )#C(( )s*#C2C CC D$C<<Dc#n Kt|}g}d}|j |j}|j}t fdt t dz Ds t dt|| D]\}} |kDr7| kr2djddj|dgg}|dz }| kr2||k\rtd ||dz|fz| |d k(s||k7rd }ntt|}|jd ||fzdjddj|dgyw) Nrc3:K|]}||dzkyw)r9Nr).0rrs r! z&COOData.encode_data..5s"EA3q6SQZ'Esr9z8liac-arff can only output COO matrices with sorted rows.r{rO}z3Instance %d has at least %d attributes, expected %dr5rJ%d %s) r>rcolrallrrArUrrrrr) rirrnum_attributesrrrrLrErs @r!rzCOOData.encode_data+sUZ hhhhyyEs3x!|1DEE,- -tS#. 0KAsC[ !C'((C();S#ABB!H1$K"C' n$I #'>:; yAGqAv!#a&) OOGsAh. /% 0(hhSXXh/566s B7D5;A:D5Nrrrrrrr r!rrs  4"7r rceZdZdZdZy)LODGeneratorDatac #PK|D][}t|}t|ts t |j Dcic]\}}||dn |||c}}]ycc}}w#t $r}dt |vr td}~wt$r t|wxYwwr) r`rrr\rrArrrru)rirrrr]rr{rs r!rzLODGeneratorData.decode_rowsPs )C"3'Ffd+k! ))/9%3EMDz#u7MM99  ) 9 C(+-- )#C(( )s:+B&A*A$A*!B&$A** B#3B  B##B&c#Kd}t|}|D]}g}t|dkDr+t||k\rtd|t|dz|fzt|D]@}||}| |dk(s||k7rd} nt t |} |j d|| fzB|dz }djdd j|d gyw) Nrrr9r5rJrrrrOr)r>rrrrrrr) rirrrrrrrrLrEs r!rzLODGeneratorData.encode_dataas Z ;CH3x!|CN :@ #c(Q,?@ c{ 4H9R16A%c!f-A3( 23  4 1 K((C(!3S9: :% ;sC C Nrrr r!rrOs )";r rc eZdZy)LODDataNrrr r!rryrr rc|tk(r tS|tk(r tS|tk(r t S|t k(r tS|tk(r tStdt|z)NzMatrix type %s not supported.) DENSErCOOrLODr DENSE_GENrLOD_GENrrAr) matrix_types r!_get_data_object_for_decodingr}siev  y  y  !!##  !!83{;KKLLr ct|dr$|jdk(r tStdt |dt r t StS)Nr)coozCannot guess matrix format!r)hasattrr)rrArrrr)matrixs r!_get_data_object_for_encodingrsIvx ==E !9 :; ; F1It $yv r c@eZdZdZdZdZdZdZdefdZ defdZ y ) ArffDecoderzAn ARFF decoder.c g|_d|_y)z Constructor.rN) _conversors _current_linerhs r!rjzArffDecoder.__init__sr c4tjdd|}|S)a(INTERNAL) Decodes a comment line. Comments are single line strings starting, obligatorily, with the ``%`` character, and can have any symbol, including whitespaces or special characters. This method must receive a normalized string, i.e., a string without padding, including the " " characters. :param s: a normalized string. :return: a string with the decoded comment. z^\%( )?r5)r'rK)rirEress r!_decode_commentzArffDecoder._decode_commentsffZQ' r c|jdd\}}|j}tj|s t t |jd}|S)aF(INTERNAL) Decodes a relation line. The relation declaration is a line with the format ``@RELATION ``, where ``relation-name`` is a string. The string must start with alphabetic character and must be quoted if the name includes spaces, otherwise this method will raise a `BadRelationFormat` exception. This method must receive a normalized string, i.e., a string without padding, including the " " characters. :param s: a normalized string. :return: a string with the decoded relation name. rr9"')splitstrip _RE_RELATIONrDror)rirE_rLrs r!_decode_relationzArffDecoder._decode_relationsPwwsA1 GGI!!!$#% %!''%.! r c|jdd\}}|j}tj|}|s t |j \}}t |jd}|dddk(rA|dddk(r9 t|jd}t|tr t||fSt |j}|d vr t||fS#t$r twxYw) a(INTERNAL) Decodes an attribute line. The attribute is the most complex declaration in an arff file. All attributes must follow the template:: @attribute where ``attribute-name`` is a string, quoted if the name contains any whitespace, and ``datatype`` can be: - Numerical attributes as ``NUMERIC``, ``INTEGER`` or ``REAL``. - Strings as ``STRING``. - Dates (NOT IMPLEMENTED). - Nominal attributes with format: {, , , ...} The nominal names follow the rules for the attribute names, i.e., they must be quoted if the name contains whitespaces. This method must receive a normalized string, i.e., a string without padding, including the " " characters. :param s: a normalized string. :return: a tuple (ATTRIBUTE_NAME, TYPE_OR_VALUES). rr9rNrrHrz{} r ) rr _RE_ATTRIBUTErDrsgroupsrr` Exceptionrrrupper)rirErrLmnametype_s r!_decode_attributezArffDecoder._decode_attributes6wwsA1 GGI    "$& &hhj e4::e$% !9 bc c 1 )%ekk%&89%&&((e} J$$&EDD&((e} )&(( )s C((C<Fcd_ttr0jdj ddj dddggd}i}t |}t}tD]}xjdz c_|jd}|s-|j} | jtr)|tk(r |d xxj|dzz cc<{| jtr.|tk7r tt}j||d <| jtr|tk7r|tk7r tt}j!|} | d|vrt#| d|| dj|| d<|d j%| t| dt&t(fr |rt+| d} n+t-| d} ntd t.t.d } | | d} j0j%| | jt2r|tk7r tn$| jt4stfd} |j7| j0|d<|d j9dr |d dd|d <|S)zDo the job the ``encode``.rz z r0r5)rrrrr9z rrrc*tt|Srl)rCfloat)xs r!z%ArffDecoder._decode..Gs#eAh-r )rrr r c3KD]D}xjdz c_|j}|s+|jtrA|Fyw)Nr9)rr startswith _TK_COMMENT)rrEris r!rz#ArffDecoder._decode..stream_sE ""a'"iiks~~k:I  s-AAArNrH)rrrrr&rr_TK_DESCRIPTIONiterrrr _TK_RELATIONr\r  _TK_ATTRIBUTErrrrtuplerrrr_TK_DATArrendswith)rirEencode_nominalrobjattribute_namesrSTATEru_rowattrr CONVERSOR_MAPrs`` r!_decodezArffDecoder._decodes a  ((6<``, where ``relation-name`` is a string. :param name: a string. :return: a string with the encoded relation declaration.  %{},"%s"r)r)rirchars r!_encode_relationzArffEncoder._encode_relations3 Dt|d{  %d++r cdD] }||vsd|z}nt|ttfr/|Dcgc]}dt|z}}ddj |z}t d|d|Scc}w)a(INTERNAL) Encodes an attribute line. The attribute follow the template:: @attribute where ``attribute-name`` is a string, and ``datatype`` can be: - Numerical attributes as ``NUMERIC``, ``INTEGER`` or ``REAL``. - Strings as ``STRING``. - Dates (NOT IMPLEMENTED). - Nominal attributes with format: {, , , ...} This method must receive a the name of the attribute and its type, if the attribute type is nominal, ``type`` must be a list of values. :param name: a string. :param type_: a string or a list of string. :return: a string with the encoded attribute declaration. r3r4rwz{%s}z, r)rrrrrr)rirrr5type_ktype_tmps r!_encode_attributezArffEncoder._encode_attributesy. Dt|d{  eeT] +CHI}V44IHIDIIh/0E($66JsA&cj|j|Dcgc]}|}}dj|Scc}w)zEncodes a given object to an ARFF file. :param obj: the object containing the ARFF information. :return: the ARFF file as an string. r0) iter_encoder)rir#rrs r!encodezArffEncoder.encodes5 $//4555yy6s 0c#K|jddr,|djdD]}|j||jds td|j |dd|jds tdt }|dD]}t |ttfr!t|d k7st |d tstd t|zt |d tr"|d tvrGtd t|zt |d ttfstd t|z|d |vrtdt|d z|j|d |j|d |d d|d}td|vrCt|jd}|j!|jd|Ed{dy7 w)aThe iterative version of `arff.ArffEncoder.encode`. This encodes iteratively a given object and return, one-by-one, the lines of the ARFF file. :param obj: the object containing the ARFF information. :return: (yields) the ARFF file as strings. rNr0rz.Relation name not found or with invalid value.r5rzAttributes not found.r8rz"Invalid attribute declaration "%s"r9zInvalid attribute type "%s"z6Trying to use attribute name "%s" for the second time.r)getrr1rr6rrrrr>r _SIMPLE_TYPESaddr:r rr)rir#rr$r'rrs r!r<zArffEncoder.iter_encodes 77=$ '=)//5 0**3// 0wwz"LM M##C O44ww|$34 4% % ;DdUDM24yA~d1gs+ DSY NOO$q'3'7-/#$A#d)$KLL Q%7 =c$i GHHAw/)!/14T!W!>?? ##DG,((a$q': :/ ;0&  S=0AD''D D D EsGH? H Nr) rrrrqr1r6r:r=r<rr r!r/r/s&", 7D;r r/Fc>t}|j|||S)a(Load a file-like object containing the ARFF document and convert it into a Python object. :param fp: a file-like object. :param encode_nominal: boolean, if True perform a label encoding while reading the .arff file. :param return_type: determines the data structure used to store the dataset. Can be one of `arff.DENSE`, `arff.COO`, `arff.LOD`, `arff.DENSE_GEN` or `arff.LOD_GEN`. Consult the sections on `working with sparse data`_ and `loading progressively`_. :return: a dictionary. r"r+rr-)fpr"r+decoders r!loadrGs(mG >>"^&1  33r c>t}|j|||S)aConvert a string instance containing the ARFF document into a Python object. :param s: a string object. :param encode_nominal: boolean, if True perform a label encoding while reading the .arff file. :param return_type: determines the data structure used to store the dataset. Can be one of `arff.DENSE`, `arff.COO`, `arff.LOD`, `arff.DENSE_GEN` or `arff.LOD_GEN`. Consult the sections on `working with sparse data`_ and `loading progressively`_. :return: a dictionary. rCrD)rEr"r+rFs r!loadsrI's(mG >>!N&1  33r ct}|j|}t|}|D]}|j|dz|}|j||S)zSerialize an object representing the ARFF document to a given file-like object. :param obj: a dictionary. :param fp: a file-like object. r0)r/r<rRwrite)r#rEencoder generatorlast_rowrs r!dumprO9s[mG##C(IIH D!HHX Ir c8t}|j|S)zSerialize an object representing the ARFF document, returning a string. :param obj: a dictionary. :return: a string with the ARFF document. )r/r=)r#rLs r!dumpsrQKs mG >># r )Trq __author____author_email__ __version__r'rStypingrrrrrrr r r@rrrrr r(UNICODErr rrrYrPrrtyping_extensionsrrrr-rVrZr?rrBrupdaterrFrMr`rrrrr_SUPPORTED_DATA_STRUCTURESr rdrorsrurrrrrrr\rrrrrrrrrrrrrrr/rGrIrOrQ)rr_rLs000r!rZs\6tjG - DDD9    2::?L 2::F S "**4bjjA2::?@"**. ; bjjrzz:TN49%+;I;S#X/f+;*<''          499=aSVX\)=?+@+@+BC41a!Q$CD"E"I>q CF*>? "1<  #S#y'B&I&2 233 M 1}1  }    m   = 5 5 1]1L L , ) )";%;%|==  >- =7=7~(;(;T n.  M hhVKK`"u3$"u3$$[>C>s$K K K