K im&dZddlZddlZddlmZddlmZmZddlZddl Z ddl m Z ddl mZmZmZdZej$j'dgd d Zej*d Zej$j'd gd dZdZdZdZdZdZej$j'dddZej$j'dddZej$j'dddgdZej$j'dgddZ ej$j'dd d!Z!d"Z"ej$j'd#e#e$fd$Z%ej$jMexrejNjPd%kd&'ej$j'd#e jRd(e jRd)fd*Z*d+Z+d,Z,d-Z-d.Z.d/Z/ej$j'd0gd1ej$j'd2gd1d3Z0d4Z1d5Z2d6Z3d7Z4ej$j'd#d8d9Z5ej$j'd:d;e jRd<fd=e jRd>fgd?Z6d@Z7dAZ8ej$jMexrejNjPd%kd&'ej$j'd#e jrdBej$judCdDZ;ej$jMexrejNjPd%kd&'ej$j'd#e jrdBej$judCdEZz2test_converters_negative_indices..AJ"&&E!Hrr!r"r r$r%rr=r converters)rrrrrr?rr(convrrs r test_converters_negative_indicesrs_ . /C < =Dxx#sc266]S#J?@H **S cd KChrcZtd}ddi}tjddgdtjgddgg}tj|tj d |d dg }t ||tjtd d d dgdd i}t|d dgd dggy)Nz$1.5,2.5,3.5 3.0,4.0,XXX 5.5,6.0,7.5 r;cB|dk(rtjSt|Srrrs rrz?test_converters_negative_indices_with_usecols..rrr!g @r r$g@rr)r=rrusecolsz0,1,2 0,1,2,3,4cy)Nr;)r]s rrz?test_converters_negative_indices_with_usecols..sr)rrr)rrrrrr?rr rs r-test_converters_negative_indices_with_usecolsrs < =C < =Dxx#sc266]S#J?@H ** jjB  Ch **X45W"|1E GCsaWq"g./rcgd}tjtd5tj|ddddy#1swYyxYw)N)1,2,3rz4,3,2,1z2the number of columns changed from 3 to 4 at row 3rxrrr})rowss rtest_ragged_errorrs< (D zF H( 43'(((s AA c`td}tjddgddgddgg}tj|tdddg}t ||td}t jtd5tj|tdddgdddy#1swYyxYw) N$0,0,XXX 0,XXX,0,XXX 0,XXX,XXX,0,XXX rrr=rrz0,0,XXX 0 0,XXX,XXX,0,XXX z/invalid column index -2 at row 2 with 1 columnsrx) rrrrrgrr~rrr(rrs rtest_ragged_usecolsrs < =Cxx!Q!Q!Q01H **Sq"g FCh 2 3C zG IE 3esQGDEEEs ; B$$B-ctd}tj|tjgdg}|jdk(sJ|jtjgk(sJy)Nrrr)rD)rrrr=rr(rs rtest_empty_usecolsr sU < =C **S R HC 99   99 $$ $rc1)r)uのu🫕c2c:td|d|d}tj|tjdd}tjd|dj dd|dj dgtjd }t ||y) Nza,z ,c,1.0 e,z,2.0,gU12rrfz,c,1.0ze,r<)rrrr=rsplitr)rrr(rrs rtest_large_unicode_charactersrs Rt:bT0 1C **S3 ?Cxx bT  s #r"V_%:%:3%?@hhuoHhrctd}ddi}tj|tjd|dd}tjddgd d gd d gg}t ||y) Nucat,dog αβγ,δεζ abc,def rc"|jSNupperrs rrz-test_unicode_with_converter..!s rrrr=rrencodingCATdoguΑΒΓuδεζABCdef)rrrr=rrr(rrrs rtest_unicode_with_converterrsk 6 7C " #D ** hhuo  Cxx%(H)=u~NOHhrctd}tjdtjfdtjfdg}ddd}tj ||d| }tj gd | }t||y) Nz$1.5,2.5,Abc 3.0,4.0,dEf 5.5,6.0,ghI mr)r_U8c0tdt|zS)Nrq)rhrgrs rrz6test_converter_with_structured_dtype..0sR%(]+rc"|jSrrrs rrz6test_converter_with_structured_dtype..0s 1779r)rr;rr))r"r)rr#DEF)7r%GHIr<)rrr=int32float32rrr)r(rjrrrs r$test_converter_with_structured_dtyper-sq < =C C?S"**$5~F GB+1D ED **Scd CCxx>bHhrctd}tj}tj|tj d|dd}tj ddgdd gg}t||y ) aD With the 'bytes' encoding, tokens are encoded prior to being passed to the converter. This means that the output of the converter may be bytes instead of unicode as expected by `read_rows`. This test checks that outputs from the above scenario are properly decoded prior to parsing by `read_rows`. zabc,def rst,xyzU3rbytesrrrRSTXYZN)rrrrrr=rrrs r!test_converter_with_unicode_dtyper8sa % &C ;;D ** rxx~$# Cxx%%89Hhrcddz}|dddz}t|dz}tj|dt}t |tj dd gd y) Nz 1.5, 2.5,Pr; rrrVr!r")rr)rrrrgrtile)rowr(rs rtest_read_huge_rowrJsT  C cr(T/C 37 C **SCu 5CbggsCj*56redfgFDGcd}tj|}tj|g|d}||jdk(sJy)Na0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.123456789r<rz 0.123456789)rr=rtype)r=fieldvalues rtest_huge_floatrRsD &E HHUOE JJwe ,R 0E EJJ}- -- -r) given_dtypeexpected_dtypeSS5UU5ctd}tj||d}tjddgddgdd gg| }t ||t |j |y ) z The given dtype is just 'S' or 'U' with no length. In these cases, the length of the resulting dtype is determined by the longest string found in the file. zAAA,5-1 BBBBB,0-3 C,4-9 rrfAAAz5-1BBBBBz0-3Cz4-9r<N)rrrrrr=)rrr(rrs rtest_string_no_length_givenr [sb 0 1C **S s ;Cxx '5)C<8HhN+rcgd}tdj|}tj|}tj|Dcgc] }t |c}}t ||ycc}w)z Some tests that the conversion to float64 works as accurately as the Python built-in `float` function. In a naive version of the float parser, these strings resulted in values that were off by an ULP or two. )z0.9999999999999999z9876543210.123456z5.43215432154321e+300z0.901z0.333rN)rjoinrrrrgr)stringsr(rrrs rtest_float_conversionrqsW G 499W% &C **S/Cxx73aq34Hh4sA-ctd}tj|td}|jtk(sJt |ddgddggt |j tjddgddggy)Nz 1, 0 10, -1rrfTFrCr)rrrboolr=r viewuint8rs r test_boolrsk > "C **S 4C 99  sdE]T4L9:sxx)QFQF+;.gens8q (ACqQqA' ' (s$&rrf)rrr)rCrrC)rroro)rDrr)rrrhrrr+rrs rtest_read_from_generatorr-s>( **SU# 5CxxIy)DEHhrcd}tj|dd}tjgdd}t||y)Nc3DKtdD]}|d|dz yw)NrDrror'r)s rr+z/test_read_from_generator_multitype..gens-q !ACqQ.  !s zi, drrf))rg)rCg?)rr r<)rrrrr,s r"test_read_from_generator_multityper0s8! **SU&C 8Cxx7vFHhrcd}tjtd5tj|dddddy#1swYyxYw)Nc3(KgdEd{y7w)N)z1,2s3, 5i1rrrrr+z)test_read_from_bad_generator..gens***s &non-string returned while reading datarxzi, irrfr)r+s rtest_read_from_bad_generatorr4sC+  F H7 35#6777s AAzPython lacks refcountsctdfd}tddz}tjtd5t j |td|idddtjdk(sJy#1swY$xYw) Nrc2dkDr tddz S)Nizfailed half-way through!rC)r)r] already_readsentinels rrz/test_object_cleanup_on_read_error..convs& $ 78 8 rzx 'zat row 5000, column 1rxr=rr) objectrr~rrrrsys getrefcount)rr(r7r8s @@r!test_object_cleanup_on_read_errorr>suxHL 55= !C z)@ A< 3f!T;< ??8 $ )) )<@r<rNrW) rrr=r?r~rrrseekrr )r(r=rrs rtest_quote_support_defaultra-s ) *C HHmdBJJ%7$ 9KL ME zB D4 3es34 HHQKxx/0>H **S DCsH%44s CCctd}d}tjt|5t j |dddddy#1swYyxYw)N1,2 3,4z,.*must be a single unicode character or Nonerxrz''rrXrr~rrrr)r(rs rtest_quotechar_multichar_errorrf>sF : C 9C y ,7 3#6777s A  Actd}d}tjt|5t j |ddddddtjt|5t j |ddd gddddt j |dd d }t |d d gddggy#1swYtxYw#1swYAxYw)NrczWwhen multiple comments or a multi-character comment is given, quotes are not supported.rxr123rN)rr rXr%)rrOrCrrDro)rr~rrrrr)r(rrs r'test_comment_multichar_error_with_quoterjGs : C $ z -F 3#EF z -K 3#c cJK **SC&C HC1v1v&'FFKKsB6&C6B?C c(td}tjdtjfdtjfddtj fg}tj gd|}tj||dd }t||y) Nz{1000;2.4;'alpha';-34 2000;3.1;'beta';29 3500;9.9;'gamma';120 4090;8.1;'delta';0 5001;4.4;'epsilon';-99 6543;7.8;'omega';-1 r,r-r.r1r2r<rFrOrW) rrr=r>r?r@rrr )rr=rrs r!test_structured_dtype_with_quotesrlWs  $ D HH  T2::. tRWWoN Exx  H **T# ECsH%rctd}tjgdd}tj|ddd}t ||y)Nz 1 "4" ""14r"U1r<rrNrr=rXrrrrrrs rtest_quoted_field_is_not_emptyrtts: > "Cxxd3H **SCts CChrctd}tjgdd}tj|ddd}t ||y)Nz1 "4" "rnrqr<rrNrrrsrs r(test_quoted_field_is_not_empty_nonstrictrvzs< = !Cxxd3H **SCts CChrctd}tjdd}tj|ddd}t ||y)Nz"Hello, my name is ""Monty""!"zHello, my name is "Monty"!U40r<rrNrWrsrs r"test_consecutive_quotechar_escapedrys< 3 4Cxx4EBH **S DChrr)r"z z# 1 2 3 # 4 5 6 r)rrCrrrCrrDc$|d}n |dk(rd}nd}t|}tjtd5t j |||}dddj |k(sJtd 5}|j||jd tjtd5t j |||}ddd|j |k(sJ dddy#1swYxYw#1swY0xYw#1swYyxYw) zDCheck that a UserWarning is emitted when no data is read from input.N)rrDrrR)rinput contained no datarx)rrw)moder) rr~warns UserWarningrrrrwriter`)rrrexpected_shaper(rfhs rtest_warn_on_no_datars  ! 4.C k)B C<jjE7;< 99 && &  %+   \\+-F G @**Sw?C @yyN*** ++ << @ @++s/C.4>D2C: D.C7:D ?DDrScd}t|}tjtd5t j ||dddy#1swYyxYw)Nz 1 2 3 4 5 6r|rx)rB)rr~rrrr)rBrr(s rtest_warn_on_skipped_datarsB D 4.C k)B C+ 3*+++s A  Ar)rrC)u2rC)i4)u4r)i8  )u8r)float16g#{?)rg. 7) complex64y. 7`F$:)r?g complex128ywfUD3":) longdoubler clongdoubleyp`P@0 BB)U2u𐈃򠬌swapTFc(tj|}d|dg}|r|j}tjdd|fgd}|jdddk(sJtj||dd }|d|j |k(sJy) Nzx,r)r)S1rKFrmrCrr)rr= newbyteorderfieldsrr)r=rrrfull_dtrs rtest_byteswapping_and_unalignedrs( HHUOErN D ""$hh c5\2%@G >># q !Q && & **TC !C s8uzz%( (( (refdFD?ctd}tj||dd}t|tjddgj |y)Nu 3 ," 2 "rrNrWrDr)rrrr rastypers r!test_unicode_whitespace_strippingrsE $ %C **S DCsBHHaV,33E:;rFDcd}||jddg}tj||d}t|tjgdgdzy)Nz0 1 , 2+3j , ( 4+5j ), ( 6+-7j ) , 8j , ( 9j ) ru rrf)rCy@@y@@y@y @y"@r)r&rrr r)r=linerrs r)test_unicode_whitespace_stripping_complexrsL ?D $,,sH- .D **T# 6CsBHH&I%JQ%NOPrr)z1 +2jz1+ 2jz1+2 jz1+-+3z(1jz(1z(1+2jz1+2j)ctjt5tj|dzg|ddddy#1swYyxYw)Nrrrfr})r=rs rtest_bad_complexrs; z "? EDL>#>???s AA efgdFDGc|jdk(rtjdtjt5t j dg|dddddy#1swYyxYw)Ngz0longdouble/clongdouble assignment may misbehave.1rrNrW)lowerr~xfailrrrrr<s rtest_nul_character_errorrsX {{} GH z "I G9ESCHIIIs A&&A/cX|dk(rtjd|dvrtjdtdt dcxk(rt dcxk(rdk(sJJtj t5tjdg|dddy#1swYyxYw) Nez5half assignment currently uses Python float convertereGz2clongdouble assignment is buggy (uses `complex`?).1_1rrz1_1 r<) r~skiprrhrgcomplexrrrrr<s rtest_no_thousands_supportrs | KL } IJ u:u = =2 == == = z "+ G9E*+++s >B  B)z1,2 z2 ,3 z2 ,3 cd}tjt|5tj|ddddy#1swYyxYw)Nz7Found an unquoted embedded newline within a single linerxrrr})rrs rtest_bad_newline_in_iteratorrs; DC z -( 43'(((s ?Az2,3 z'2 ',3 z'2 ',3 z'2 ',3 cXtj|dd}t|ddgddggy)NrrOrdrbrcr rG)rrs rtest_good_newline_in_iteratorrs- **TSC 8Csb"XBx01rnewline)r  cgd}|Dcgc]}|jd|}}tj|tdd}t |dd|gdd |dggycc}w) N)z1,"2 " z3,"4 z1" rrrNrWro23rp)r&rrr;r )rrrrs rtest_universal_newlines_quotedr sg +D26 73CKKg & 7D 7 **T3# FCscQwi=1C1WIQ3HIJ 8sActjddgd}t|gdgdgtjddgd t }|j gd gd gk(sJy) Nz123 z456rrz)rorprz1,2,3 z4,5,6rrV)rz2r)z4z56)rrr r;tolist)rs rtest_null_characterr+s^ **o}5 HCsY 23 **'9"& 2C ::<35LM MM MrcGdd}tjtd5tj|t ddddy#1swYyxYw)NceZdZdZdZy):test_iterator_fails_getting_next_line..BadSequencecy)Nrarselfs r__len__zBtest_iterator_fails_getting_next_line..BadSequence.__len__8src6|dk(r td|d|dzS)N2Bad things happened!z, rC) RuntimeError)ritems r __getitem__zFtest_iterator_fails_getting_next_line..BadSequence.__getitem__;s*rz"#9::V2dQhZ( (rN)__name__ __module__ __qualname__rrrrr BadSequencer7s   )rrrrxrrf)r~rrrrrh)rs r%test_iterator_fails_getting_next_liner6sD)) |+A B< ;=s;<< : > HH & & : :  ; > > > > AA--A6cGdd}tjtd5tjj j |tjdddddy#1swYyxYw)NceZdZdZdZy)BTestCReaderUnitTests.test_filelike_read_fails..BadFileLikercb|xjdz c_|jdkDr tdy)NrC Bad bad bad!z1,2,3 )counterrrsizes rreadzGTestCReaderUnitTests.test_filelike_read_fails..BadFileLike.readSs* ! <<"$&~66 rNrrrrrrrr BadFileLikerPs G !rrrrxr*Tr)r~rrrrrrr=rrs rtest_filelike_read_failsz-TestCReaderUnitTests.test_filelike_read_failsLsd ! !]]<~ > C HH & & : : RXXc]T ; C C C C AA44A=cGdd}tjtd5tjj j |tjdddddy#1swYyxYw)NceZdZdZdZy)@TestCReaderUnitTests.test_filelike_bad_read..BadFileLikercy)Nirrs rrzETestCReaderUnitTests.test_filelike_bad_read..BadFileLike.readesrNrrrrrrbs G rrr3rxr*Tr)r~rrrrrrr=rs rtest_filelike_bad_readz+TestCReaderUnitTests.test_filelike_bad_read]sh    ]]9BD C HH & & : : RXXc]T ; C C C Crctjtd5tjj j ttjdddddy#1swYyxYw)Nz/error reading from object, expected an iterablerxr*Fr r~rrrrrrr;r=rs rtest_not_an_iterz%TestCReaderUnitTests.test_not_an_itermsW ]]9KM ? HH & & : :  ; ? ? ? ?rctjtd5tjj j tdddddy#1swYyxYw)Nzinternal error: dtype mustrxr*Fr)r~rrrrrrr;rs r test_bad_typez"TestCReaderUnitTests.test_bad_typessL ]]9,H I 5 HH & & : :e ; 5 5 5 5s 5AA#ctjtd5tjj j ttjddddddy#1swYyxYw)Nzencoding must be a unicoderxr*F{)r=rrrrs rtest_bad_encodingz&TestCReaderUnitTests.test_bad_encodingxs[ ]]9,H I M HH & & : :  ; M M M Ms A A..A7r)rrrctdjd|d}tjjj |tj ddddd }t|ddd fd d |ddgy)Nz0 1 "2 " 3 4 # rr")rU10TrNrrC)r=rquoter skiplinesrrorrz4 )rr&rrrrr=r )rrrrs rtest_manual_universal_newlinesz3TestCReaderUnitTests.test_manual_universal_newlines}s{/77gF "$hh((<< $sa=1 3q!t9sayM3&EFrN) rrrrrrrrrr~mark parametrizerrrrrrDsK> C"C ? 5 M  [[Y(<= G> Grrctjtd5tjt ddddddy#1swYyxYw)N".*control characters.*incompatiblerx1, 2, 3rrrDrrr'test_delimiter_comment_collision_raisesrs? y(L ME 8I&#DEEE "AActjtd5tjt ddddddy#1swYyxYw)NrrxrrrdrDrrr)test_delimiter_quotechar_collision_raisesr s? y(L MF 8I&#EFFFrctjtd5tjt ddddddy#1swYyxYw)Nrrx1 2 3r)r rXrDrrr'test_comment_quotechar_collision_raisesr s? y(L MC 8G$scBCCCrctjtd5tjt ddddgdddy#1swYyxYw)Nz0Comment characters.*cannot include the delimiterrxrrrrrDrrr5test_delimiter_and_multiple_comments_collision_raisesrsH K L 8I&#c KLLLs $A  Acxtjtd5tjt d|d|d|d|d |dddtjtd5tjt d|d|d|d|d | dddy#1swY[xYw#1swYyxYw) Nrrxrorz3 45z6 r rXrD)rs r,test_collision_with_default_delimiter_raisesrs y(L MI 8at1RDRD"S9:RHI y(L MJ 8at1RDRD"S9:bIJJIIJJs.B$-.B0$B-0B9nl)rrctd|d|d||d|d|d||}d}tjt|5t j || dddtjt|5t j || dddtjt|5t j || dddy#1swYxYw#1swYQxYw#1swYyxYw) Nrorrrprrz&control character.*cannot be a newlinerxrrrre)rr(rs r%test_control_character_newline_raisesrs Qrd!B4qRD"Qrd!B4t< =C 2C y ,& 3"%& y ,% 3$% y ,& 3"%&& &&%%&&s#C:C"5C.C"C+.C7) generic_data long_datumunitless_dtyper)z2012-03z 2013-01-15M8zM8[D]) spam-a-lottis_but_a_scratchrU17nrows)rqr`c |g|z|gz}tj||}t||dzk(sJt|t|k(sJtdj |}tj ||}t|t|k(sJ|j |j k(sJt||t\} } tj| t| d5} | jdj |dzdddtj | |}t|t|k(sJ|j |j k(sJt||tj | |t|dz }tj| t|t|dz k(sJt||dt|dz y#1swYxYw)zCheck that the correct unit (e.g. month, day, second) is discovered from the data when a user specifies a unitless datetime.r<rCrr}Nr)r=rv)rrlenrrrr=rroscloseopenrrhremove) rrrrrrrr(r)fdfnamers rtest_parametric_unit_discoveryr)s >E !ZL 0DxxN3H t9 !! ! t9H %% % 499T? #C 3n-A q6S] "" " 77hnn $$ $H IBHHRL eS )R 44'() 5/A q6S] "" " 77hnn $$ $H 5UQYHAIIe q6S^ ## #H_c%!)n-.))s -$G!!G*c|dgdzdgz}tjdgdzdgzd}d}tdj|}tj|d | }|j |j k(sJt ||t\}}tj|t|d 5}|jdj|dddtj|d | }tj||j |j k(sJt ||y#1swY^xYw) Nrr XXXtis_but_a_scratchrrr<c$|jdSr) removeprefixrs rrz>test_str_dtype_unit_discovery_with_converter..sQ^^E*rrrr:r}) rrrrrr=rrr#r$r%rr&)rrrr(r)r'r(rs r,test_str_dtype_unit_discovery_with_converterr.s >E !%;$< !C**C3nskip)rr9i90ri3 ic d}djdtd|dzD}td||z }tj|dz|dz|zj t }t|}tj|dd|d}t||k(sJt||ddd f|d z }|j|t |} tj| dd|d}t||k(sJt||ddd fy) Nirc3&K|] }|d yw)z a 0.5 1Nr).0r*s r zEtest_skiprow_exceeding_maxrows_exceeding_chunksize..;sGs(^GsrCr r9r)r=rrBrvrz test_data.txt) rr(minrarangerr9rrr"r r) tmpdirr<r:rexpected_lengthrr(rtmp_filer(s r2test_skiprow_exceeding_maxrows_exceeding_chunksizerF6sK 99GU1kAo-FG GD%u!45OyyEAI$?@GGLH 4.C **Suu UC s8 && &xQT+'H NN4 ME **U%3QV WC s8 && &xQT+r)n__doc__r#r<iortempfilerrr~numpyrnumpy.ma.testutilsr numpy.testingrrr rrrr*fixturerArHrNrYr[rkrtrrrrrrrgr;rskipifimplementationversionr=rrrrrrrrrrrrr rr typecodesfilterwarningsrrrrr$r-r0r4r>r@rErGrJrLr[r]rarfrjrlrtrvryrrreprrrrrrrrrrrrrrr r rrrr)r.r0r4r6r;rFrrrrTs3   0 +CC B $LM:N:!!@\2/3/*O@ LL":;-<- ":;.<. #NN#CD,E,      5&/2O3OGG 2 2 : :i G9;8288D>8288D>"BC4D;4 0&( E%3434 55     $7),.-.% hbhhtn hbhhtn , , &=GG 2 2 : :i G9;",,|"<=NO 4P>; 4GG 2 2 : :i G9;",,|"<=NO4P>;4 2<<"?@ 6  7A    7 $-EF*G*&GG 2 2 : :i G9;%;%!eWb!12:3:GG 2 2 : :i G9;K;K0818 < o. &/ &o. &/ &&"GG 2 2 : :i G9;7;7 ( &:   !EF),T9$56+7-G+*V,+-+'7+."$>!K+ tJKL ) GHI&!.()"$/ )0#)$ ) \"W,s24<4<$'Q(QGG 2 2 : :i G9;$'KM?M(; ? GG 2 2 : :i G9; LL & 2S 8:I:;IGG 2 2 : :i G9; \"Y.46 +6; + i i"(( i k k m " 2  2 $89K:KN <FGFGRE F C L J J|,&-&F07"45 /6 /F0GG 2 2 : :i G9;:;: 12!3!!=>?("IJ,K,r