i ^dZddlmZddlmZmZddlmZmZddl m Z m Z m Z ddl m Z ddlZddlmZmZmZmZmZmZddlZddlZdd lmZdd lmZdd lmZmZdd l m!Z!m"Z"dd l#m$Z$ddl%m&Z&ddl'm(Z(ddl)m*Z*m+Z+m,Z,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3ddl4m5Z5m6Z6ddl7m8Z8ddl9m:Z:ddl;mcm?Z@ddl=mAZAddlBmCZCddlDmEZEddlFmGZGer,ddlHmIZImJZJmKZKmLZLddlMmNZNddlOmPZPmQZQmRZRddlSmTZTmUZUmVZVdd l2mWZWd!ZX ddd'ZYdd+ZZ ddd1Z[ ddd4Z\dddd-d5dd7Z]e dddDZ^e dddGZ^e$dHddd,dddej_fddMZ^e dddPZ`e dddQZ`e$dHdd,ddddej_fddRZ`e dddTZae dddUZae$dHdd,ddddej_dfddVZa dddbZbdddcZcecZd dddfZeGdgdhe<ZfGdideeTr>)rLr@r>) isinstancedictr) issubclassdtypetypenpfloatingintegerr)colr>r@s r;_handle_date_columnrXps&$M 3))&))) > sy~r{ 3 3 #)."*55 F @ @ @s8&cJJJ J  ? 3 3 Ms--- -s8FLLL Lr= data_framer"returncDt|}t|D]n\}\}}t|jt s||vrH ||}n#t tf$rd}YnwxYw||t||o|S)zz Force non-datetime columns to be read as such. Supports both string formatted and integer timestamp columns. Nr@) r< enumerateitemsrOrRrKeyError TypeErrorisetitemrX)rYr:icol_namedf_colfmts r;_parse_date_columnsrfs 0 <!>LL Hf flO 4 4 LK8O8O !(+i(       #6vc#J#J#J K K K sA!!A76A7Tnumpy coerce_float dtype_backendDtypeBackend | Literal['numpy']ctj|}|jd}tt |jd||}|dkrt d}g}|D]m} || d} | jdkr'| | } | t| n|}|r%tj||t|dSt| S) Nr)rRrhripyarrowT) from_pandasstringF)columnsindexverify_integrity)ro)rto_object_array_tuplesshaper(listTrarrayrRrrnappendr$r" _from_arraysrange) datarorhricontentidx_lenarrayspa result_arraysarrpa_arrays r;_convert_arrays_to_dataframers (..GmAG ! WY!# F  !! ' 2 2  @ @Cxxx66HyH$$$==55  !4X!>!> ? ? ? ? *% G5>>E    ))))r=rRDtypeArg | Nonect||||}|r||}t||}|||}|Sz5Wrap result set of a SQLAlchemy query in a DataFrame.)rastyperf set_index)rzro index_colrhr:rRriframes r; _wrap_resultrs[ )w m T TE $ U## { 3 3E ** Lr=)rr:rRridfc|r||}t||}|||}|Sr)rrfr)rrr:rRris r;_wrap_result_adbcrsH  YYu   R - -B \\) $ $ Ir=. table_namestrrstr | list[str] | Noner:=list[str] | dict[str, str] | dict[str, dict[str, Any]] | Nonerolist[str] | None chunksizeNoneDtypeBackend | lib.NoDefaultc dSN rconschemarrhr:rorris r;read_sql_tabler r=intIterator[DataFrame]c dSrrrs r;rr #r=pandasr str | None int | NoneDataFrame | Iterator[DataFrame]c lt||tjurd}|tjusJt||d5} | |st d|d| |||||||} dddn #1swxYwY| | St d|d|)at Read SQL database table into a DataFrame. Given a table name and a SQLAlchemy connectable, returns a DataFrame. This function does not support DBAPI connections. Parameters ---------- table_name : str Name of SQL table in database. con : SQLAlchemy connectable or str A database URI could be provided as str. SQLite DBAPI connection mode not supported. schema : str, default None Name of SQL schema in database to query (if database flavor supports this). Uses default schema if None (default). index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Can result in loss of Precision. parse_dates : list or dict, default None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. columns : list, default None List of column names to select from SQL table. chunksize : int, default None If specified, returns an iterator where `chunksize` is the number of rows to include in each chunk. dtype_backend : {'numpy_nullable', 'pyarrow'} Back-end data type applied to the resultant :class:`DataFrame` (still experimental). If not specified, the default behavior is to not use nullable data types. If specified, the behavior is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` :class:`DataFrame` .. versionadded:: 2.0 Returns ------- DataFrame or Iterator[DataFrame] A SQL table is returned as two-dimensional data structure with labeled axes. See Also -------- read_sql_query : Read SQL query into a DataFrame. read_sql : Read SQL query or database table into a DataFrame. Notes ----- Any datetime values with time zone information will be converted to UTC. Examples -------- >>> pd.read_sql_table("table_name", "postgres:///db_name") # doctest:+SKIP rgTrneed_transactionzTable z not foundrrhr:rorriN)rr no_defaultpandasSQL_builder has_table ValueError read_table) rrrrrhr:rorri pandas_sqltables r;rrs'^ &&&&&  . . . . 3v E E E  ##J// >>>sABBBparams$list[Any] | Mapping[str, Any] | Nonec dSrr sqlrrrhrr:rrRris r;read_sql_queryrvrr=c dSrrrs r;rrrr=c t||tjurd}|tjusJt|5} | ||||||||cdddS#1swxYwYdS)a Read SQL query into a DataFrame. Returns a DataFrame corresponding to the result set of the query string. Optionally provide an `index_col` parameter to use one of the columns as the index, otherwise default integer index will be used. Parameters ---------- sql : str SQL query or SQLAlchemy Selectable (select or text object) SQL query to be executed. con : SQLAlchemy connectable, str, or sqlite3 connection Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Useful for SQL result sets. params : list, tuple or mapping, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype : Type name or dict of columns Data type for data or columns. E.g. np.float64 or {'a': np.float64, 'b': np.int32, 'c': 'Int64'}. dtype_backend : {'numpy_nullable', 'pyarrow'} Back-end data type applied to the resultant :class:`DataFrame` (still experimental). If not specified, the default behavior is to not use nullable data types. If specified, the behavior is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` :class:`DataFrame` .. versionadded:: 2.0 Returns ------- DataFrame or Iterator[DataFrame] Returns a DataFrame object that contains the result set of the executed SQL query, in relation to the specified database connection. See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql : Read SQL query or database table into a DataFrame. Notes ----- Any datetime values with time zone information parsed via the `parse_dates` parameter will be converted to UTC. Examples -------- >>> from sqlalchemy import create_engine # doctest: +SKIP >>> engine = create_engine("sqlite:///database.db") # doctest: +SKIP >>> sql_query = "SELECT int_column FROM test_data" # doctest: +SKIP >>> with engine.connect() as conn, conn.begin(): # doctest: +SKIP ... data = pd.read_sql_query(sql_query, conn) # doctest: +SKIP rg)rrrhr:rrRriN)rrrr read_query) rrrrhrr:rrRrirs r;rrsr &&&&&  . . . . 3    :$$ %#'%                      sA))A-0A- list[str]c dSrr rrrrhrr:rorrirRs r;read_sqlrs r=c dSrrrs r;rr s #r=c t||tjurd}|tjusJt|5} t | t r)| |||||||| cdddS | |} n#t$rd} YnwxYw| r(| |||||||cdddS| |||||||| cdddS#1swxYwYdS)ag Read SQL query or database table into a DataFrame. This function is a convenience wrapper around ``read_sql_table`` and ``read_sql_query`` (for backward compatibility). It will delegate to the specific function depending on the provided input. A SQL query will be routed to ``read_sql_query``, while a database table name will be routed to ``read_sql_table``. Note that the delegated function might have more specific notes about their functionality not listed here. Parameters ---------- sql : str or SQLAlchemy Selectable (select or text object) SQL query to be executed or a table name. con : ADBC Connection, SQLAlchemy connectable, str, or sqlite3 connection ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. The user is responsible for engine disposal and connection closure for the ADBC connection and SQLAlchemy connectable; str connections are closed automatically. See `here `_. index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table (only used when reading a table). chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype_backend : {'numpy_nullable', 'pyarrow'} Back-end data type applied to the resultant :class:`DataFrame` (still experimental). If not specified, the default behavior is to not use nullable data types. If specified, the behavior is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` :class:`DataFrame` .. versionadded:: 2.0 dtype : Type name or dict of columns Data type for data or columns. E.g. np.float64 or {'a': np.float64, 'b': np.int32, 'c': 'Int64'}. The argument is ignored if a table is passed instead of a query. .. versionadded:: 2.0.0 Returns ------- DataFrame or Iterator[DataFrame] Returns a DataFrame object that contains the result set of the executed SQL query or an SQL Table based on the provided input, in relation to the specified database connection. See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql_query : Read SQL query into a DataFrame. Notes ----- ``pandas`` does not attempt to sanitize SQL statements; instead it simply forwards the statement you are executing to the underlying driver, which may or may not sanitize from there. Please refer to the underlying driver documentation for any details. Generally, be wary when accepting statements from arbitrary sources. Examples -------- Read data from SQL via either a SQL query or a SQL tablename. When using a SQLite database only SQL queries are accepted, providing only the SQL tablename will result in an error. >>> from sqlite3 import connect >>> conn = connect(":memory:") >>> df = pd.DataFrame( ... data=[[0, "10/11/12"], [1, "12/11/10"]], ... columns=["int_column", "date_column"], ... ) >>> df.to_sql(name="test_data", con=conn) 2 >>> pd.read_sql("SELECT int_column, date_column FROM test_data", conn) int_column date_column 0 0 10/11/12 1 1 12/11/10 >>> pd.read_sql("test_data", "postgres:///db_name") # doctest:+SKIP For parameterized query, using ``params`` is recommended over string interpolation. >>> from sqlalchemy import text >>> sql = text( ... "SELECT int_column, date_column FROM test_data WHERE int_column=:int_val" ... ) >>> pd.read_sql(sql, conn, params={"int_val": 1}) # doctest:+SKIP int_column date_column 0 1 12/11/10 Apply date parsing to columns through the ``parse_dates`` argument The ``parse_dates`` argument calls ``pd.to_datetime`` on the provided columns. Custom argument values for applying ``pd.to_datetime`` on a column are specified via a dictionary format: >>> pd.read_sql( ... "SELECT int_column, date_column FROM test_data", ... conn, ... parse_dates={"date_column": {"format": "%d/%m/%y"}}, ... ) int_column date_column 0 0 2012-11-10 1 1 2010-11-12 .. versionadded:: 2.2.0 pandas now supports reading via ADBC drivers >>> from adbc_driver_postgresql import dbapi # doctest:+SKIP >>> with dbapi.connect("postgres:///db_name") as conn: # doctest:+SKIP ... pd.read_sql("SELECT int_column FROM test_data", conn) int_column 0 0 1 1 rg)rrrhr:rrirRNFr) rrrrrOSQLiteDatabaserr Exceptionr) rrrrhrr:rorrirRr_is_table_names r;rrsx &&&&&  . . . . 3  ': j. 1 1 ((#)'#+)  '''''''' #'11#66NN # # #"NNN #  ((#)'#+))''''''''<((#)'#+)  =''''''''''''''''''s;2C:?BC: B$!C:#B$$ C:C::C>C>failautoname if_exists3Literal['fail', 'replace', 'append', 'delete_rows']rp index_labelIndexLabel | Nonemethod"Literal['multi'] | Callable | Noneenginec T|dvrtd|dt|tr|}n$t|tst dt ||d5} | j||f||||||| | d| cdddS#1swxYwYdS) a+ Write records stored in a DataFrame to a SQL database. .. warning:: The pandas library does not attempt to sanitize inputs provided via a to_sql call. Please refer to the documentation for the underlying database driver to see if it will properly prevent injection, or alternatively be advised of a security risk when executing arbitrary commands in a to_sql call. Parameters ---------- frame : DataFrame, Series name : str Name of SQL table. con : ADBC Connection, SQLAlchemy connectable, str, or sqlite3 connection or sqlite3 DBAPI2 connection ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. schema : str, optional Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). if_exists : {'fail', 'replace', 'append', 'delete_rows'}, default 'fail' - fail: If table exists, do nothing. - replace: If table exists, drop it, recreate it, and insert data. - append: If table exists, insert data. Create if does not exist. - delete_rows: If a table exists, delete all records and insert data. index : bool, default True Write DataFrame index as a column. index_label : str or sequence, optional Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. chunksize : int, optional Specify the number of rows in each batch to be written at a time. By default, all rows will be written at once. dtype : dict or scalar, optional Specifying the datatype for columns. If a dictionary is used, the keys should be the column names and the values should be the SQLAlchemy types or strings for the sqlite3 fallback mode. If a scalar is provided, it will be applied to all columns. method : {None, 'multi', callable}, optional Controls the SQL insertion clause used: - None : Uses standard SQL ``INSERT`` clause (one per row). - ``'multi'``: Pass multiple values in a single ``INSERT`` clause. - callable with signature ``(pd_table, conn, keys, data_iter) -> int | None``. Details and a sample callable implementation can be found in the section :ref:`insert method `. engine : {'auto', 'sqlalchemy'}, default 'auto' SQL engine library to use. If 'auto', then the option ``io.sql.engine`` is used. The default ``io.sql.engine`` behavior is 'sqlalchemy' **engine_kwargs Any additional kwargs are passed to the engine. Returns ------- None or int Number of rows affected by to_sql. None is returned if the callable passed into ``method`` does not return an integer number of rows. Notes ----- The returned rows affected is the sum of the ``rowcount`` attribute of ``sqlite3.Cursor`` or SQLAlchemy connectable. If using ADBC the returned rows are the result of ``Cursor.adbc_ingest``. The returned value may not reflect the exact number of written rows as stipulated in the `sqlite3 `__ or `SQLAlchemy `__ )rreplacerw delete_rows'' is not valid for if_existsz9'frame' argument should be either a Series or a DataFrameTr)rrprrrrRrrN)rrOr#to_framer"NotImplementedErrorrto_sql) rrrrrrprrrRrr engine_kwargsrs r;rrs3pDDDDYDDDEEE%      y ) ) ! G    3v E E E   z     #                         s8BB!$B!ct||5}||cdddS#1swxYwYdS)a Check if DataBase has named table. Parameters ---------- table_name: string Name of SQL table. con: ADBC Connection, SQLAlchemy connectable, str, or sqlite3 connection ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). Returns ------- boolean rN)rr)rrrrs r;rrXs* 3v . . .0*##J//000000000000000000s 488r PandasSQLc&ddl}t||js|t|St dd}t|t r |t dtdd|2t|t |jj frt|||St dd}|r$t||jrt|Stj d tt t|S) z Convenience function to return the correct PandasSQL subclass based on the provided parameters. Also creates a sqlalchemy connection and transaction if necessary. rN sqlalchemyignore)rLz"Using URI string without version 'z%' or newer of 'sqlalchemy' installed.zadbc_driver_manager.dbapizpandas only supports SQLAlchemy connectable (engine/connection) or database string URI or sqlite3 DBAPI2 connection. Other DBAPI2 objects are not tested. Please consider using SQLAlchemy. stacklevel)sqlite3rO Connectionrrr ImportErrorrr Connectable SQLDatabase ADBCDatabasewarningswarn UserWarningr)rrrrradbcs r;rrts8NNN#w)**#ckc"""+LJJJJ#s 2 ),1G ) ) )   *S3 8I8U2V"W"W3(8999 %&A( S S SD ! 300!C    M D #%%  #  r=ceZdZdZ d5d6dZdZd7dZd8dZd8dZd9dZ d9dZ d:dZ d;dd*Z d?d@d,Zd-Zd.Zd/Z dAdBd0ZdCd3Zd4ZdS)DSQLTablez For mapping Pandas tables to SQL tables. Uses fact that table is reflected by SQLAlchemy to do better type conversions. Also holds various flags needed to avoid having to pass them between functions all the time. NTrrrrrpbool | str | list[str] | NonerrprefixrRrrZrc ||_||_||_||_||||_||_||_| |_| |_ || |_ n*|j |j|j|_ |j td|dt|jstddS)NzCould not init table 'rzEmpty table name specified)rpd_sqlrr _index_namerprrkeysrR_create_table_setupr get_tablerlen) selfrpandas_sql_enginerrprrrrrrRs r;__init__zSQLTable.__init__s '   %%e[99  "   1133DJJ..ty$+FFDJ : =d===>> >49~~ ;9:: : ; ;r=cL|j|j|jSr)rrrrrs r;existszSQLTable.existss{$$TY <<>>$$&&&&&8++=00 '' 4;????? !QT^!Q!Q!QRRR  " " " " "r=rrrcfd|D}|j|j|}|jS)a< Execute SQL statement inserting data Parameters ---------- conn : sqlalchemy.engine.Engine or sqlalchemy.engine.Connection keys : list of str Column names data_iter : generator of list Each item contains a list of values to be inserted c Ng|]!}tt|d"ST)strictrPzip.0rowrs r; z,SQLTable._execute_insert..0GGGcSs400011GGGr=)rexecuterinsertrowcount)rconnr data_iterrzresults ` r;_execute_insertzSQLTable._execute_insertsLHGGGYGGG$$TZ%6%6%8%8$??r=cddlm}fd|D}||j|}|j|}|jS)a  Alternative to _execute_insert for DBs support multi-value INSERT. Note: multi-value insert is usually faster for analytics DBs and tables containing a few columns but performance degrades quickly with increase of columns. r)rc Ng|]!}tt|d"Sr rrs r;rz2SQLTable._execute_insert_multi..rr=)rrrvaluesrrr)rrrrrrzstmtrs ` r;_execute_insert_multizSQLTable._execute_insert_multisl &%%%%%GGGGYGGGvdj!!((..$$T**r="tuple[list[str], list[np.ndarray]]c|ji|jd}|j|j_ |dn,#t $r}t d||d}~wwxYw|j}t tt|j }t|}dg|z}t| D]\}\}}|j jdkrt|jt"rhddl} | j|j jr!|jt.} n|jj} n|j} n|j jd krr|j} t| t"r(| t5j d } | d t.} n|jt.} t| t4jsJt=| |jrtA| } d| | <| ||<||fS) NF)deepTinplacez!duplicate name in index/columns: Mr)rRrGzm8[ns]i8)!rprcopynames reset_indexrrtmaprrorr]r^rRkindrO_valuesr$rltypesis_date pyarrow_dtypeto_numpyobjectdt to_pydatetimerTviewrndarrayrS _can_hold_nar ) rtemperr column_namesncols data_listrb_serr~rEvalsmasks r; insert_datazSQLTable.insert_data sU : !:???..D#zDJ  U   .... U U U !JS!J!JKKQTT U:DCT\2233 L!!(,fun $TZZ\\22  KAx3y~$$ck+>?? 4((((x'' (?@@;K00v0>>F0022: 1133AA3&&{d$788C==rx/A/A=BBDIIdOO**622K&&v..a,, 5 5d1gg 5 5, Aww$IaLLY&&sA A.A))A.rrrrc| ||j}n@|dkr|j}n2t|rt||}nt d||\}}t |j}|dkrdS||}n|dkrt d||zdz}d}|j 5} t|D]W} | |zt| dz|z|  krn3t fd|Dddi} || || } | || }R|| z }Xdddn #1swxYwY|S) NmultizInvalid parameter `method`: rz%chunksize argument should be non-zeroc3,K|]}|VdSrr)rrend_istart_is r; z"SQLTable.insert..`s,>>Sc'%-(>>>>>>r=r T) rr callabler rrArrrrryminr)rrr exec_insertrr<nrowschunkstotal_insertedrrb chunk_iter num_insertedrFrGs @@r;rzSQLTable.insert;s >.KK w  4KK f   F!&$//KKDFDDEE E**,,iDJ A::1  II !^^DEE E9$) [ ( ( * * 7d6]] 7 7i-QUi/77e##E >>>>>I>>>GK  +{4zBB +%-)5&,6! 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7"s=A(D11D58D5rg exit_stackrrhr?rirjGenerator[DataFrame]c#rKd}|5 ||} | s|stjg||Vnbd}t| ||||_||||j!|j|jd|jV ddddS#1swxYwYdS)z,Return generator through chunked result set.FTrorhr:riNr$) fetchmanyr" from_recordsrr_harmonize_columnsrpr) rrrQrrorhr:ri has_read_datarzs r;_query_iteratorzSQLTable._query_iteratorksQ  ! ! !'' 22('4l $ 9'< '' +=(:)J((T(BBBj   + ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !sBB,,B03B0rc |ddlm}|gt|dkrTfd|D}j9jdddD](} |djj| )||} n|j} j| } | } | | ||| |||S| } t| | ||_ ||j!j jdj S) Nr)selectc4g|]}jj|Sr)rc)rnrs r;rz!SQLTable.read..s!555DJLO555r=)rhr:rirUTr$)rr\rrprrr^rrrrZfetchallrrrXr)rrQrhr:rorrir\colsidx sql_selectrr:rzs` r;readz SQLTable.reads &%%%%%  3w<>>: r=c|dur|jjj}|?t|ts|g}t ||krt d||S|dkr"d|jjvr|jjjdgStj |jjj St|tr|gSt|tr|SdS)NTz@Length of 'index_label' should match number of levels, which is rDrp) rrpnlevelsrOrtrrrorcomfill_missing_namesr)r)rrprrgs r;rzSQLTable._index_names D==j&.G&!+t440#.-K{##w..$6,366#"1 4:#555J$)1y -dj.>.DEEEs # # 7N t $ $ L4r=cVg}jgtjD]R\}}jj|}|t ||dfS|fdt tjjDz }|S)NTcg|]A}tjj|jjdd|fdfBSNF)rrroiloc)rrb dtype_mapperrs r;rz8SQLTable._get_column_names_and_types..s]# # # #A& ' 'djoaaad6K)L)Le T# # # r=) rpr]r_get_level_valuesrwrryrro)rrncolumn_names_and_typesrb idx_labelidx_types`` r;_get_column_names_and_typesz$SQLTable._get_column_names_and_typess!# : ! )$* 5 5 P P 9'< (8(J(J1(M(MNN&--s9~~x.NOOOO# # # # # 3tz12233# # #   &%r=c| ddlm m}m}ddlm}||j} fd|D}|jIt|js |jg}n|j}||d|j dzi}| ||j p|j jj }|} ||j | g|Rd|iS)Nr)ColumnPrimaryKeyConstraintr/MetaDatac4g|]\}}}|||S))rpr)rrtypis_indexrus r;rz0SQLTable._create_table_setup..s@   #c8 F4H - - -   r=r_pkr)rrurvr/rrxrs_sqlalchemy_typerrrrwrrr) rrvr/rxrprorpkcrrrus @r;rzSQLTable._create_table_setups)          /.....!%!A!A$BW!X!X    '=    9  ** ! {y&&E49u3DEEC NN3   7 0 7xzzuTY>w>>>v>>>r=ct|}|jjD]}|j} |j|}||vr6 ||}n#t $rd}YnwxYwt |||j|<R||j}|tus|tus |tur#|tu}t |||j|<n|dkr'|tur| ||j|<ntrGt|r8t!|j|r| ||j|<ng|dkrat#||kr<|t'jdus |t*ur| ||j|<#t,$rYwxYwdS)a Make the DataFrame's column types align with the SQL table column types. Need to work around limited NA value support. Floats are always fine, ints must always be floats if there are Null values. Booleans are hard because converting bool column with None replaces all Nones with false. Therefore only convert bool if there are no NA values. Datetimes should already be converted to np.datetime64 if supported, but here we also force conversion if required. Nr\rNrgint64)r<rrorrr`rX _get_dtyperSr rrfloatrrrrrcountrTrRr?r_) rr:risql_colrcrdrecol_typer>s r;rXzSQLTable._harmonize_columns s 4K@@ z)( ( G|H& H-{**#)(3$###"#+>vc+R+R+RDJx( ??7<88((4''?22#o5C+>v3+O+O+ODJx(("g--(e2C2C+1==+B+BDJx((&(( G'11 G( 8(<== G ,2==+B+BDJx(("g--#f++2O2O28G#4#444D8H8H/5}}X/F/F 8,    O( ( s;F1AF1 AF1AF1.EF11 F?>F?rWIndex | Seriesc|jpi}t|r+tt|}|j|vr ||jSt j|d}ddlm}m }m }m }m }m } m} m} m} m} |dvrK |jj |dSn/#t($r"t+|dd|dcYSYnwxYw|S|dkr*t-jd t0t3 |S|d kr#|jd kr | d S| dS|dkrw|jjdvr| S|jjdvr| S|jjdkrt7d|S|dkr|S|dkr|S|dkr| S|dkrt7d| S)NTskipnar) TIMESTAMP BigIntegerBooleanDateDateTimeFloatInteger SmallIntegerTextTime) datetime64r )timezonetz timedelta64lthe 'timedelta' type is not supported, and will be written as integer values (ns frequency) to the database.rrUfloat32) precision5rV)int8uint8int16)uint16int32uint64z1Unsigned 64 bit integer datatype is not supportedbooleanrr complexComplex datatypes not supported)rRrrrPrr infer_dtypesqlalchemy.typesrrrrrrrrrrr3rAttributeErrorgetattrrrrrlowerr)rrWrRrrrrrrrrrrrs r;r}zSQLTable._sqlalchemy_typeFs**    'u%%Ex5  SX&?3t444  1 1 1 469($9d3333)! 4 4 43d++7$9d33333387 4 O } $ $ ML+--         # #yI%%ur****ur****  " "y~##%%)CCC##%%''+>>>%%''833 !TUUU!!  " "N   K   K  " ">?? ? s7B'B<;B<cddlm}m}m}m}m}m}m}t||rtSt||rtj dSt||r|j stStSt||rtSt||rtSt||rt St||r(t#rt%tjSt(S)Nr)rrrrrrStringr)na_value)rrrrrrrrrOrrTrRrr rrr?rr%nanr2) rsqltyperrrrrrrs r;rzSQLTable._get_dtypes<                   gu % % 4L  ) ) 48G$$ $  + + 4# " "  * * 4O  & & 4K  ) ) 4K  ( ( 4!## 4"BF3333 r=)NTrrNNNN) rrrprrrrrrRrrZrrZrrZr)rrrZr)rZr!)NN)rrrrrZr)TNrg) rQrrrrhr?rirjrZrR)TNNNrg) rQrrhr?rrrirjrZr)Nrg)rirjrZr)rWr)__name__ __module__ __qualname____doc__rrrrrrr rArrZrerrsrrXr}rrr=r;rrs/3IO !%";";";";";H===EEEE 4444 ####  "/'/'/'/'f!%59.....l"9@"!"!"!"!"!N" $9@-----^@ & & &???@9@:::::xEEEEN     r=rceZdZdZd.dZd/dZ d0d1dZe d0d2dZe d3d4d%Z ed5d6d'Z ed5d7d(Z e d8d9d-Z dS):rz9 Subclasses Should define read_query and to_sql. rZrc|Srrrs r; __enter__zPandasSQL.__enter__s r=rcdSrrrargss r;__exit__zPandasSQL.__exit__s r=NTrgrrrrrhr?rrrrrirjrc trr) rrrrhr:rorrris r;rzPandasSQL.read_tables "!r=rrRrc dSrr) rrrrhr:rrrRris r;rzPandasSQL.read_querys  r=rrrrrrprrrc  dSrr) rrrrrprrrrRrrrs r;rzPandasSQL.to_sqls  r=str | Select | TextClausecdSrr)rrrs r;rzPandasSQL.execute r=cdSrr)rrrs r;rzPandasSQL.has_tablerr=rr"rrcdSrrrrrrrRrs r;_create_sql_schemazPandasSQL._create_sql_schemas  r=)rZrrNTNNNNrgrrrrrhr?rrrrrirjrZrrrrrrhr?rrrRrrirjrZrrTNNNNNr)rrrrrpr?rrrRrrrrrrZrrrrrrrrrZr?NNN rr"rrrrrRrrrrZr) rrrrrrrrrrrrrrr=r;rrsm    -1!! $9@ " " " " "-1! $!%9@     ^   JP $!%59    ^     ^     ^  "&!%!     ^   r=c eZdZ d dd ZdS) BaseEngineTNrrrrrprrrrZc   t|)z: Inserts data into already-prepared table )r) rrrrrrprrrrs r;insert_recordszBaseEngine.insert_recordss"$'''r=TNNN rrrrrprrrrZr)rrrrrr=r;rrs;04 $(((((((r=rc(eZdZddZ dddZdS)SQLAlchemyEnginerZrc(tdddS)Nrz'sqlalchemy is required for SQL support.)extra)rrs r;rzSQLAlchemyEngine.__init__s)"  I      r=TNrrrrrprrrc  ddlm}  |||S#| j$rB} d} t | j} t j| | rtd| | d} ~ wwxYw)Nr)exc)rrzg(\(1054, "Unknown column 'inf(e0)?' in 'field list'"\))(?# )|inf can not be used with MySQLzinf cannot be used with MySQL) rrrStatementErrorrorigresearchr)rrrrrrprrrrrr9msgerr_texts r;rzSQLAlchemyEngine.insert_recordss #""""" <<)F<CC C!   0C38}}Hyh'' K !@AAsJI s A+=A&&A+rrr)rrrrrrr=r;rrsO    04 $r=rc0|dkrtd}|dkrYtg}d}|D]:} |cS#t$r}|dt|zz }Yd}~3d}~wwxYwtd||dkrtSt d) zreturn our implementationrz io.sql.enginez - NzUnable to find a usable engine; tried using: 'sqlalchemy'. A suitable version of sqlalchemy is required for sql I/O support. Trying to import the above resulted in these errors:rz*engine must be one of 'auto', 'sqlalchemy')r!rrrr)rengine_classes error_msgs engine_classr9s r; get_enginer/s O,, *+ * 1 1L 1#|~~%%% 1 1 1gC00  1        !!! A B BBs 7 A AA c"eZdZdZ d>d?d Zd@d Zed ZdAdBdZ dCdDdZ e dEdFd#Z dCdGd$Z e Z dHdId,ZdJd-Z dKdLd2Zed3ZdAdMd4ZdAdNd6ZdAdOd7ZdAdOd8Z dPdQd=ZdS)Rraa This class enables conversion between DataFrame and SQL databases using SQLAlchemy to handle DataBase abstraction. Parameters ---------- con : SQLAlchemy Connectable or URI string. Connectable to connect with the database. Using SQLAlchemy makes it possible to use any DB supported by that library. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). need_transaction : bool, default False If True, SQLDatabase will create a transaction. NFrrrr?rZrcddlm}ddlm}ddlm}t |_t|tr*||}|j |j t||r,|j | }|r@|s,|j |||_|||_d|_dS)Nr) create_engine)EnginerwrF)rrsqlalchemy.enginerrrxrrQrOrcallbackdispose enter_contextconnectin_transactionbeginrrreturns_generator)rrrrrrrxs r;rzSQLDatabase.__init__as -,,,,,,,,,,,...... $++ c3   2-$$C O $ $S[ 1 1 1 c6 " " ?/// >>C  7C$6$6$8$8 7 O ) )#))++ 6 6 6HF+++ !&r=cJ|js|jdSdSr)rrQclosers r;rzSQLDatabase.__exit__xs1% $ O ! ! # # # # # $ $r=c#K|js=|j5|jVddddS#1swxYwYdS|jVdSr)rrrrs r;rzSQLDatabase.run_transaction|sx&&(( !!  h                  (NNNNNs A  AAr"str | Select | TextClause | Deletecddlm}|gn|g}t|tr |jj}n |jj} ||g|RS#|$r}td|d||d}~wwxYw)z,Simple passthrough to SQLAlchemy connectabler)SQLAlchemyErrorNExecution failed on sql '': )sqlalchemy.excrrOrrexec_driver_sqlrr)rrrrrexecute_functionrs r;rzSQLDatabase.executes222222^rr& c3   0#x7  #x/  T##C/$/// / T T T IC I IC I IJJPS S TsAA& A!!A&Trgrrrrrhrrrirjrc |j|j|gdt||||} |d|_| |j|||||S)a Read SQL database table into a DataFrame. Parameters ---------- table_name : str Name of SQL table in database. index_col : string, optional, default: None Column to set as index. coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. This can result in loss of precision. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg}``, where the arg corresponds to the keyword arguments of :func:`pandas.to_datetime`. Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table. schema : string, default None Name of SQL schema in database to query (if database flavor supports this). If specified, this overwrites the default schema of the SQL database object. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype_backend : {'numpy_nullable', 'pyarrow'} Back-end data type applied to the resultant :class:`DataFrame` (still experimental). If not specified, the default behavior is to not use nullable data types. If specified, the behavior is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` :class:`DataFrame` .. versionadded:: 2.0 Returns ------- DataFrame See Also -------- pandas.read_sql_table SQLDatabase.read_query T)ronlyviews)rprN)rhr:rorri)rreflectrrrrerQ) rrrrhr:rorrrirs r;rzSQLDatabase.read_tablesy@ txzl$GGGT6JJJ  %)D "zz O%#'    r=rQrrrRrrRc #Kd} |5 ||} | s| stg||||||Vnd} t| ||||||VM ddddS#1swxYwYdS)+Return generator through chunked result setFTrrhr:rRriN)rVr) rrQrrorrhr:rRrirYrzs r;rZzSQLDatabase._query_iterators    '' 22 ( *#&/)5(3"'*7 $ "'!- +"/!                   sAA$$A(+A(c |||} | } |*d|_|| |j|| ||||| S| } t | | |||||} | S)a Read SQL query into a DataFrame. Parameters ---------- sql : str SQL query to be executed. index_col : string, optional, default: None Column name to use as index for the returned DataFrame object. coerce_float : bool, default True Attempt to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'} parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype : Type name or dict of columns Data type for data or columns. E.g. np.float64 or {'a': np.float64, 'b': np.int32, 'c': 'Int64'} Returns ------- DataFrame See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql NTr)rrrrZrQrar) rrrrhr:rrrRrirrorzrs r;rzSQLDatabase.read_query snc6**++--  %)D "''#)'+(   ??$$D #)'+ELr=rrrrrprrc |rt|st||}ntt|}ddlm}|D]N\} } t| trt| |r+t| |r`. engine : {'auto', 'sqlalchemy'}, default 'auto' SQL engine library to use. If 'auto', then the option ``io.sql.engine`` is used. The default ``io.sql.engine`` behavior is 'sqlalchemy' **engine_kwargs Any additional kwargs are passed to the engine. )rrrrprrrR)rrrrrprrr)rrr)rr rrr)rrrrrprrrrRrrr sql_enginerrNs r;rzSQLDatabase.to_sqlsz '' #   32         !!tF!;;;r=c|jjSr)rtablesrs r;rzSQLDatabase.tabless yr=crddlm}||j}|||p |jjS)Nrr )rrrrrr)rrrrrs r;rzSQLDatabase.has_tablesD<<<<<<!!$(++~~dF$>di.>???r=r/cddlm}m}|p |jj}|||j|j|}|jD]#}t|j|r d|j_ $|S)Nr)Numericr/) autoload_withrF) rrr/rrrrorOrS asdecimal)rrrrr/tblcolumns r;rzSQLDatabase.get_tables        +49+eJ &QQQk . .F&+w// .(- % r=c|p |jj}|||r|j|j|g|d|5||||jdddn #1swxYwY|jdSdS)NTrrrrr) rrrrrrrdropclearrrrs r;r zSQLDatabase.drop_tables+49+ >>*f - -  I  XZLt    %%'' G Gz62277TX7FFF G G G G G G G G G G G G G G G IOO        s0BB Bcj|p |jj}|||r|j|j|g|d|||}|||j dSdS)NTr") rrrrrrrdeleterr$)rrrrs r;rzSQLDatabase.delete_rows%s+49+ >>*f - -  I  XZLt    NN:v66E LL ( ( . . 0 0 0 IOO        r=rr"rrc pt|||d|||}t|SNF)rrprrRr)rrrrrrrrRrrs r;rzSQLDatabase._create_sql_schema/sI     5##%%&&&r=rl)rrrr?rZrrr)rrrrNTNNrg) rQrrrrhr?rRrrirjrZrRr)rTNNN) rrrrrprrRrrZrrrrrrZrrrrrrrpr?rrrrrRrrrrrrZrr)rrrrrZr/rrrrrZrrr)rrrrrrrrrr staticmethodrZrrr rrpropertyrrrr rrrr=r;rrOsZ$HM'''''.$$$$^ T T T T T$-1!! $9@K K K K K Z !!%9@&&&&\&V-1! $!%9@RRRRRhH JP/3!%.....`BJP! $!%59VVVVVp  X @@@@@     "&!%! '''''''r=rceZdZdZd.dZedZd/d0d Z d1d2dZ d1d3dZ e Z d4d5d&Z d/d6d'Z d/d7d(Z d8d9d-ZdS):rz This class enables conversion between DataFrame and SQL databases using ADBC to handle DataBase abstraction. Parameters ---------- con : adbc_driver_manager.dbapi.Connection rZrc||_dSrrrrs r;rzADBCDatabase.__init__P r=c#K|j5} |Vn(#t$r|jwxYw|jddddS#1swxYwYdSr)rcursorrrollbackcommitrcurs r;rzADBCDatabase.run_transactionSs X__   #     !!###  HOO                      s%A1#A1%AA11A58A5Nrrc ddlm}t|tst d|gn|g}|j} |j|g|R|S#|$r^} |jn&#|$r}td|d|d}||d}~wwxYwtd|d|}||d}~wwxYw NrErrorz/Query must be a string unless using sqlalchemy.zExecution failed on sql:  z unable to rollbackrr) adbc_driver_managerr?rOrr`rr7rr8r rrrr?rr;r inner_excexs r;rzADBCDatabase.execute]s+------#s## OMNN N^rr&hoo  CK #d # # # #J    (!!#### ( ( ("PPPsPPPi'  ( H3HH3HHIIB#  6 AB?"A<;B:<BBBB::B?Trgrrrrrhr?rrrrrirjrc |durtd|rtd|r9|rt|} ng} | |z} dd| D} nd} |r d| d|d |} nd| d|} || 5} | }t || }d d d n #1swxYwYt ||| S) a Read SQL database table into a DataFrame. Parameters ---------- table_name : str Name of SQL table in database. coerce_float : bool, default True Raises NotImplementedError parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg}``, where the arg corresponds to the keyword arguments of :func:`pandas.to_datetime`. Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table. schema : string, default None Name of SQL schema in database to query (if database flavor supports this). If specified, this overwrites the default schema of the SQL database object. chunksize : int, default None Raises NotImplementedError dtype_backend : {'numpy_nullable', 'pyarrow'} Back-end data type applied to the resultant :class:`DataFrame` (still experimental). If not specified, the default behavior is to not use nullable data types. If specified, the behavior is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` :class:`DataFrame` .. versionadded:: 2.0 Returns ------- DataFrame See Also -------- pandas.read_sql_table SQLDatabase.read_query T2'coerce_float' is not implemented for ADBC drivers/'chunksize' is not implemented for ADBC drivers, c3"K|] }d|dV dS)"Nr)rxs r;rHz*ADBCDatabase.read_table..s*#@#@HHHH#@#@#@#@#@#@r=*zSELECT z FROM .riN)rr:)rr'joinrfetch_arrow_tabler*r)rrrrhr:rorrri index_select to_select select_listrr;pa_tablers r;rzADBCDatabase.read_tablessv t # #%D   Y%&WXX X   ".y99 ! $w.I))#@#@i#@#@#@@@KKK  =E[EEEEEEDD<[<< <z1'index_label' is not implemented for ADBC driversrHz+'dtype' is not implemented for ADBC driversz,'method' is not implemented for ADBC driversrz1engine != 'auto' not implemented for ADBC driversrNrrrrr DROP TABLE rwr)preserve_indexzdatatypes not supportedN)rrzmodedb_schema_namez"Failed to insert records on table=z with mode=)rrAr?rrrrrrr/rmArrowNotImplementedErrorrr7 adbc_ingestrr9)rrrrrprrrrRrrrr~r?rrZ sql_statementrrr;rNs r;rzADBCDatabase.to_sql sX( 2 2------  %C   Y%&WXX X  U%&STT T  V%&TUU U V  %C   "++T++JJJ  >>$ ' ' /F"" !H:!H!H!HIIIi'' :j : :  ]++113333h&&m++  v... A(&&uU&CCCC* A A A677S @ AX__   # !$##D"1""   #LLLTLL                 sND$$ E.D>>EF+!E;:F+;FFFF++F/2F/c|j||}|dD])}|s|D]!}|s|dD]}|d|krdS"*dS)N)db_schema_filtertable_name_filtercatalog_db_schemasdb_schema_tablesrTF)radbc_get_objectsread_all to_pylist)rrrrcatalog_schema schema_record table_records r;rzADBCDatabase.has_tablez sx((#t)  (** ##78BBDD $ $N! !/ $ $ $$12D$E$$L#L1T99#tttt:$  $ur=c|r|d|n|}|||r,|d|dSdS)NrN DELETE FROM )rrr)rrrrs r;rzADBCDatabase.delete_rows sl+1;'''''t >>$ ' ' > LL4 44 5 5 ; ; = = = = = > >r=rr"rrc td)Nznot implemented for adbcrrs r;rzADBCDatabase._create_sql_schema s""<===r=rrrrrrrr-rr,rr)rrrrrrrrrrrrrrrrr=r;rrFsj^2-1!! $9@X X X X X z-1! $!%9@C C C C C JH JP! $!%59dddddL$>>>>>"&!%! >>>>>>>r=rTEXTREALINTEGERrDATETIME)rnrUrVr rr rr2c t|ddd}n&#t$r}t d|d|d}~wwxYw|S)Nzutf-8r z%Cannot convert identifier to UTF-8: 'r)rencodedecode UnicodeErrorr)runamer9s r;_get_unicode_namerw swSD   (33::7CC SSSHHHHIIsRS Ls69 AAAct|}t|std|d}|dkrtdd|ddzdzS)Nz$Empty table or column name specifiedrz%SQLite identifier cannot contain NULsrKz"")rwrrfindr)rrv nul_indexs r;_get_valid_sqlite_namer| sv d # #E u::A?@@@ 6""IA~~@AAA sD)) )C //r=c`eZdZdZdfd ZddZddZddZdd Zdd Z dd Z dZ dZ xZ S) SQLiteTablezw Patch the SQLTable for fallback support. Instead of a table variable just use the Create Table statement. rZrcbtj|i||dSr)superr_register_date_adapters)rrkwargs __class__s r;rzSQLiteTable.__init__ s7$)&))) $$&&&&&r=c(ddl}d d}d}d}|t||t||t|d}d}|d ||d |dS) NrrZrcL|jdd|jdd|jdd|jdS)N02d:rN06d)hourminutesecond microsecond)ts r; _adapt_timez8SQLiteTable._register_date_adapters.._adapt_time s:fTTT18TTT!(TTTTTT Tr=c*|Sr isoformatvals r;z5SQLiteTable._register_date_adapters.. sS]]__r=c,|dS)N rrs r;rz5SQLiteTable._register_date_adapters.. ss););r=cNtj|Sr)r fromisoformatrtrs r;rz5SQLiteTable._register_date_adapters.. s4#5cjjll#C#Cr=cNtj|Sr)r rrtrs r;rz5SQLiteTable._register_date_adapters.. s(>szz||(L(Lr=r timestampr)rregister_adapterr rr register_converter)rrradapt_date_isoadapt_datetime_iso convert_dateconvert_timestamps r;rz#SQLiteTable._register_date_adapters s  U U U U54;;  {333  ~666  +=>>>CC LL""6<888"";0ABBBBBr=rcPtd|jS)Nz; )rrPrrs r;rzSQLiteTable.sql_schema s5::dj))***r=c|j5}|jD]}|| ddddS#1swxYwYdSr)rrrr)rr;rs r;rzSQLiteTable._execute_create s [ ( ( * * "c  " " D!!!! " " " " " " " " " " " " " " " " " " "s AA A num_rowsrc ttt|jj}d}t |j)|jdddD]}|d| fd|D}d|}d|gt|z d fdt|D}d |j d|d |}|S) N?r`rc&g|] }|Srr)rr escapes r;rz0SQLiteTable.insert_statement.. s!>>>f66&>>>>>r=,cg|]}dd S)()r)rr= row_wildcardss r;rz0SQLiteTable.insert_statement.. s%LLLq2-222LLLr=z INSERT INTO  (z ) VALUES ) rtr+rrror|rprrPrryr) rrr)wldrcbracketed_names col_names wildcardsinsert_statementrrs @@r;rzSQLiteTable.insert_statement s Sdj01122' : !z$$B$' % % Q$$$$>>>>>>>HH_-- #U!344 HHLLLLE(OOLLLMM O66$),, O O O OI O O  r=cddlm}t|} ||d|n#|$r}t d|d}~wwxYw|jS)Nrr>rDrzExecution failed)rr?rt executemanyrrr)rrrrr?r<rs r;rzSQLiteTable._execute_insert s!!!!!!OO  =   T22A2>> J J J J = = = 233 < =}s*AAAAct|}d|D}||t|||jS)Ncg|] }|D]}| Srr)rrrLs r;rz5SQLiteTable._execute_insert_multi.. s%>>>#>>Q!>>>>r=r)rtrrrr)rrrrr<flattened_datas r;r z!SQLiteTable._execute_insert_multi sSOO >>y>>> T**C NN*CC^TTT}r=c ||j}t fd|D}|jzt |jrft |js |jg}n|j}d fd|D}|d|jd|d|j r |j dz}nd }d |z |jzd zd |zd zg}d|D}|rd|}d fd|D}|d d|jzdz|zzdz |jzdz|zdz|S)z Return a list of SQL statements that creates a table reflecting the structure of a DataFrame. The first entry will be a CREATE TABLE statement while the rest will be CREATE INDEX statements. c:g|]\}}}|dz|zS)rr)rcnamectyper=rs r;rz3SQLiteTable._create_table_setup.. s>   ,;E5!FF5MMC % '   r=NrIc&g|] }|Srrrr^rs r;rz3SQLiteTable._create_table_setup..( s!";";";66!99";";";r=z CONSTRAINT z_pk PRIMARY KEY (rrNrz CREATE TABLE z ( z, z )cg|] \}}}|| Srr)rrr=r{s r;rz3SQLiteTable._create_table_setup..9 s$VVV/UAxXV5VVVr=r=rc&g|] }|Srrrs r;rz3SQLiteTable._create_table_setup..< s!!=!=!=&&))!=!=!=r=z CREATE INDEX ix_zON r) rs_sql_type_namer|rrrrPrwrr) rrpcreate_tbl_stmtsr cnames_br schema_name create_stmtsix_colscnamesrs @r;rzSQLiteTable._create_table_setup s# "&!A!A$BU!V!V'    ?U    9 S^^  ** ! {y ";";";";d";";";<>I   &*S069::;&##$        r=c|jpi}t|r+tt|}|j|vr ||jSt j|d}|dkr+tjdttd}n'|dkrd}n|d krd }n|d krtd |tvrd }t|S) NTrrrrrVrr emptyrnrr) rRrrrPrrrrrrrr _SQL_TYPES)rrWrRrs r;rzSQLiteTable._sql_type_nameI s**    'u%%Ex5  SX&?3t444 } $ $ ML+--      !HH  % %!HH  HH  " ">?? ? : % %H(##r=rr)rrrZr)rZr)rrrrrrrrrrr rr __classcell__)rs@r;r~r~ s '''''' CCCC:++++""""     & 111f $ $ $ $ $ $ $r=r~ceZdZdZd,dZedZd-d.d Ze d/d0dZ d1d2dZ dZ d3d4d#Z d-d5d&Z d-d6d(Zd-d7d)Zd-d7d*Z d8d9d+ZdS):rz Version of SQLDatabase to support SQLite connections (fallback without SQLAlchemy). This should only be used internally. Parameters ---------- con : sqlite connection object rZrc||_dSrr3r4s r;rzSQLiteDatabase.__init__w r5r=c#&K|j} |V|jn(#t$r|jwxYw |dS#|wxYwr)rr7r9rr8rr:s r;rzSQLiteDatabase.run_transactionz shoo III HOO        H         IIKKKKKCIIKKKKs;A:%A  A::BNrrc ddlm}t|tst d|gn|g}|j} |j|g|R|S#|$r^} |jn&#|$r}td|d|d}||d}~wwxYwtd|d|}||d}~wwxYwr=) rr?rOrr`rr7rr8rrBs r;rzSQLiteDatabase.execute s+!!!!!!#s## OMNN N^rr&hoo  CK #d # # # #J    (!!#### ( ( ("PPPsPPPi'  ( H3HH3HHIIB#  rETrgrrrhr?rRrrirjrRc #RKd} ||} t| tkrt| } | sJ||s2t jg||} |r| |} | VdSd}t| ||||||V)rFTrTrN) rVrStuplertrr"rWrr) r7rrorrhr:rRrirYrzrs r;rZzSQLiteDatabase._query_iterator s  ##I..DDzzU""Dzz  $!&3G,F6!'u!5!5 LLL M#)'+   ! r=rrc |||} d| jD} ||| || |||||S|| } | t | | |||||} | S)Ncg|] }|d S)rr)rcol_descs r;rz-SQLiteDatabase.read_query.. sBBB88A;BBBr=r)r descriptionrZ_fetchall_as_listrr) rrrrhr:rrrRrir7rorzrs r;rzSQLiteDatabase.read_query sc6**BBv/ABBB  ''#)'+(   ))&11D LLNNN #)'+ELr=cv|}t|tst|}|Sr)rarOrt)rr;rs r;rz SQLiteDatabase._fetchall_as_list s1&$'' "&\\F r=rrrrrrprrrc |rt|st||}ntt|}|D]/\} } t | t st| d| d0t|||||||}| | || S)a Write records stored in a DataFrame to a SQL database. Parameters ---------- frame: DataFrame name: string Name of SQL table. if_exists: {'fail', 'replace', 'append', 'delete_rows'}, default 'fail' fail: If table exists, do nothing. replace: If table exists, drop it, recreate it, and insert data. append: If table exists, insert data. Create if it does not exist. delete_rows: If a table exists, delete all records and insert data. index : bool, default True Write DataFrame index as a column index_label : string or sequence, default None Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. schema : string, default None Ignored parameter included for compatibility with SQLAlchemy version of ``to_sql``. chunksize : int, default None If not None, then rows will be written in batches of this size at a time. If None, all rows will be written at once. dtype : single type or dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a string. If all columns are of the same type, one single value can be used. method : {None, 'multi', callable}, default None Controls the SQL insertion clause used: * None : Uses standard SQL ``INSERT`` clause (one per row). * 'multi': Pass multiple values in a single ``INSERT`` clause. * callable with signature ``(pd_table, conn, keys, data_iter)``. Details and a sample callable implementation can be found in the section :ref:`insert method `. rz) not a string)rrprrrR) rrPr rr^rOrrr~rr)rrrrrprrrrRrrrrWr rs r;rzSQLiteDatabase.to_sql sj  H&& * eU33T5)) %  H H W!'3//H$%F%Fw%F%F%FGGGH  #     ||Iv...r=rrcd}d|d}t|||gdkS)Nrz SELECT name FROM sqlite_master WHERE type IN ('table', 'view') AND name=z ; r)rrra)rrrrquerys r;rzSQLiteDatabase.has_tableB sS    4<<v..7799::Q>>r=rcdSrrr%s r;rzSQLiteDatabase.get_tableP str=cxdt|}||dS)NrX)r|rr)rrrdrop_sqls r;r zSQLiteDatabase.drop_tableS s<?!7!=!=?? X$$&&&&&r=cdt|}|||r)||dSdS)Nrk)r|rrr)rrr delete_sqls r;rzSQLiteDatabase.delete_rowsW s\B$:4$@$@BB >>$ ' ' - LL $ $ * * , , , , , - -r=c pt|||d|||}t|Sr))r~rrr*s r;rz!SQLiteDatabase._create_sql_schema\ sI     5##%%&&&r=rrrr+) rrrhr?rRrrirjrZrRr) rhr?rrrRrrirjrZrr)rrrrrpr?rrrRrrrrrrZrrr.r,r)rrrRrrrrZr)rrrrrrrrr/rZrrrrrr rrrr=r;rrl s  ^ , !!%9@$$$$\$R! $!%9@&&&&&P  $!%59O/O/O/O/O/b ? ? ? ? ?'''''-----!%! '''''''r=rct|5}||||||cdddS#1swxYwYdS)a] Get the SQL db table schema for the given frame. Parameters ---------- frame : DataFrame name : str name of SQL table keys : string or sequence, default: None columns to use a primary key con: ADBC Connection, SQLAlchemy connectable, sqlite3 connection, default: None ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library If a DBAPI2 object, only sqlite3 is supported. dtype : dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a SQLAlchemy type, or a string for sqlite3 fallback connection. schema: str, default: None Optional specifying the schema to be used in creating the table. r3)rrRrN)rr)rrrrrRrrs r; get_schemarp s: s # # # z,, 4d%-                    s 8<<)FN)r>r?r@rA)rYr"rZr")Trg)rhr?rirjrZr"r+)rhr?rRrrirjrZr")rr"rRrrirjrZr").......)rrrrr:rrorrrrirrZr")rrrrr:rrorrrrirrZr)rrrrrrrhr?r:rrorrrrirrZr)rrrrr:rrrrRrrirrZr")rrrrr:rrrrRrrirrZr)rrrhr?rrr:rrrrRrrirrZr).......N) rrrorrrrirrRrrZr") rrrorrrrirrRrrZr)rrrhr?rorrrrirrRrrZr)NrTNNNNr)rrrrrrrpr?rrrrrRrrrrrrZrr)rrrrrZr?rl)rrrr?rZr)rrrZr)rr2rZr)NNNN)rrrRrrrrZr)sr __future__rabcrr contextlibrrr rr functoolsr rtypingr r rrrrrrgrTpandas._configr pandas._libsrpandas.compat._optionalrr pandas.errorsrrpandas.util._decoratorsrpandas.util._exceptionsrpandas.util._validatorsrpandas.core.dtypes.commonrrrrpandas.core.dtypes.dtypesrpandas.core.dtypes.missingr rr!pandas.core.apir"r#pandas.core.arraysr$pandas.core.arrays.string_r%pandas.core.baser&pandas.core.commoncorecommonrhr'"pandas.core.internals.constructionr(pandas.core.tools.datetimesr)pandas.io._utilr*collections.abcr+r,r-r.rr/sqlalchemy.sql.expressionr0r1r2pandas._typingr3r4r5r6r<rXrfrrrrrrrrr table_existsrrrrrrrrrrwr|r~rrrr=r;rs  #"""""  ------/.....444444777777 655555++++++322222222222)))))) ......CCCCCC333333111111!        CGMMMMM605< * * * * *L!5<4!5< 0  (+QT #25        (+QT #25       H(,QU $ 25.d?d?d?d?d?N ),36QT 25       ),36QT 25       H)-37QU !25.g g g g g T ), 25!       ), 25!       H)-  $ 25.!GGGGG\EK%) !15o o o o o d000002 "%%%%%PQQQQQ|QQQhH H H H H cH H H V((((((((&zBCCCC@q'q'q'q'q')q'q'q'nS>S>S>S>S>9S>S>S>r    0000$f$f$f$f$f$(f$f$f$RA'A'A'A'A'YA'A'A'N  !        r=