'VfidZddlmZddlmZmZddlmZmZddl m Z m Z m Z ddl m Z ddlZddlmZmZmZmZmZmZddlZddlZdd lmZdd lmZdd lmZdd lm Z m!Z!dd l"m#Z#ddl$m%Z%ddl&m'Z'm(Z(m)Z)m*Z*ddl+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2m3Z3ddl4m5Z5ddl6m7Z7ddl8m9Z9ddl:m;cmZ>ddl?m@Z@ddlAmBZBddlCmDZDer*ddlEmFZFmGZGddlHmIZIddlJmKZKmLZLddlMmNZNmOZOmPZPmQZQmRZRddl/mSZSd ZT ddd&ZUd'ZV ddd/ZW ddd2ZXdddd)d3dd5ZYdd6ZZe dddCZ[e dddFZ[ddd(dddej\fddKZ[e dddNZ]e dddOZ]dd(ddddej\fddPZ]e dddRZ^e dddSZ^dd(ddddej\dfddTZ^ ddd`Z_dddaZ`e`Za ddddZbGdedfe9ZcGdgdce9eZdGdhdiZeGdjdkeeZfddlZgGdmdnedZhGdodpedZidqdrdsdtdudvdsdwZjddyZkddzZlGd{d|ecZmGd}d~edZn dddZodS)z| Collection of query wrappers / abstractions to both facilitate data retrieval and to reduce dependency on DB-specific API. ) annotations)ABCabstractmethod) ExitStackcontextmanager)datedatetimetime)partialN) TYPE_CHECKINGAnyCallableLiteralcastoverload)using_string_dtype)libimport_optional_dependency)AbstractMethodError DatabaseError)find_stack_level)check_dtype_backend) is_dict_like is_list_likeis_object_dtypeis_string_dtype)DatetimeTZDtype)isna) get_option) DataFrameSeries)ArrowExtensionArray) StringDtype) PandasObject)maybe_make_list)convert_object_array) to_datetime)arrow_table_to_pandas)IteratorMapping)Table)Select TextClause)DateTimeErrorChoicesDtypeArg DtypeBackend IndexLabelSelf)IndexcF|dus||durg}nt|ds|g}|S)z3Process parse_dates argument for read_sql functionsTNF__iter__)hasattr) parse_datess r<) isinstancedictpopr( TypeError ValueError issubclassdtypetypenpfloatingintegerr)colr<r>errors r9_handle_date_columnr[ks?&$M '-jj4&@&@&LH H   "311&111z*     377u7777 > sy~r{ 3 3 #)."*55 F @ @ @s8&cJJJ J  ? 3 3 Ms--- -s8FLLL Ls AAAcDt|}t|D]n\}\}}t|jt s||vrH ||}n#t tf$rd}YnwxYw||t||o|S)zz Force non-datetime columns to be read as such. Supports both string formatted and integer timestamp columns. Nr>) r: enumerateitemsrNrTrKeyErrorrQisetitemr[) data_framer8icol_namedf_colfmts r9_parse_date_columnsrgs 0 <!>LL Hf flO 4 4 LK8O8O !(+i(       #6vc#J#J#J K K K sA!!A76A7Tnumpy coerce_float dtype_backendDtypeBackend | Literal['numpy']returnr!c ^tj|}tt|jd||}|dkrt d}g}|D]m}||d} |jdkr'| | } | t| n|}|rZtttttt!||} || _| St|S)N)rTrirjpyarrowT) from_pandasstring)columns)rto_object_array_tuplesr'listTrarrayrTrrpappendr#r!rOziprangelenrq) datarqrirjcontentarrayspa result_arraysarrpa_arraydfs r9_convert_arrays_to_dataframers( (..G ! WY!# F  !! ' 2 2  @ @Cxxx66HyH$$$==55  !4X!>!> ? ? ? ? * tCU3w<<%8%8 9 96BBCC D D  ))))r;rTDtypeArg | Nonect||||}|r||}t||}|||}|Sz5Wrap result set of a SQLAlchemy query in a DataFrame.)rastyperg set_index)rzrq index_colrir8rTrjframes r9 _wrap_resultrs[ )w m T TE $ U## { 3 3E ** Lr;)rr8rTrjrc|r||}t||}|||}|Sr)rrgr)rrr8rTrjs r9_wrap_result_adbcrsH  YYu   R - -B \\) $ $ Ir;cZtjdttt dd}|0t |t |jjfrtdt|d 5}| ||cdddS#1swxYwYdS) a Execute the given SQL query using the provided connection object. Parameters ---------- sql : string SQL query to be executed. con : SQLAlchemy connection or sqlite3 connection If a DBAPI2 object, only sqlite3 is supported. params : list or tuple, optional, default: None List of parameters to pass to execute method. Returns ------- Results Iterable zP`pandas.io.sql.execute` is deprecated and will be removed in the future version. stacklevel sqlalchemyrBrANz+pandas.io.sql.execute requires a connectionT)need_transaction) warningswarn FutureWarningrrrNstrengineEnginerQpandasSQL_builderexecute)sqlconparamsr pandas_sqls r9rrs" M 1#%%  ,LJJJJ*S3 8I8P2Q"R"REFFF 3 6 6 6/*!!#v..//////////////////s=B  B$'B$. table_namerrstr | list[str] | Noner8!list[str] | dict[str, str] | Nonerqlist[str] | None chunksizeNoneDtypeBackend | lib.NoDefaultc dSN rrschemarrir8rqrrjs r9read_sql_tabler Cr;intIterator[DataFrame]c dSrrrs r9rr!rr;r str | None int | NoneDataFrame | Iterator[DataFrame]c lt||tjurd}|tjusJt||d5} | |st d|d| |||||||} dddn #1swxYwY| | St d|d|)a0 Read SQL database table into a DataFrame. Given a table name and a SQLAlchemy connectable, returns a DataFrame. This function does not support DBAPI connections. Parameters ---------- table_name : str Name of SQL table in database. con : SQLAlchemy connectable or str A database URI could be provided as str. SQLite DBAPI connection mode not supported. schema : str, default None Name of SQL schema in database to query (if database flavor supports this). Uses default schema if None (default). index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Can result in loss of Precision. parse_dates : list or dict, default None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. columns : list, default None List of column names to select from SQL table. chunksize : int, default None If specified, returns an iterator where `chunksize` is the number of rows to include in each chunk. dtype_backend : {'numpy_nullable', 'pyarrow'}, default 'numpy_nullable' Back-end data type applied to the resultant :class:`DataFrame` (still experimental). Behaviour is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` (default). * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` DataFrame. .. versionadded:: 2.0 Returns ------- DataFrame or Iterator[DataFrame] A SQL table is returned as two-dimensional data structure with labeled axes. See Also -------- read_sql_query : Read SQL query into a DataFrame. read_sql : Read SQL query or database table into a DataFrame. Notes ----- Any datetime values with time zone information will be converted to UTC. Examples -------- >>> pd.read_sql_table('table_name', 'postgres:///db_name') # doctest:+SKIP rhTrrzTable z not foundrrir8rqrrjN)rr no_defaultr has_tablerR read_table) rrrrrir8rqrrjrtables r9rr0s'Z &&&&&  . . . . 3v E E E  ##J// >>>sABBBr$list[Any] | Mapping[str, Any] | Nonec dSrr rrrrirr8rrTrjs r9read_sql_queryrrr;c dSrrrs r9rrrr;c t||tjurd}|tjusJt|5} | ||||||||cdddS#1swxYwYdS)a Read SQL query into a DataFrame. Returns a DataFrame corresponding to the result set of the query string. Optionally provide an `index_col` parameter to use one of the columns as the index, otherwise default integer index will be used. Parameters ---------- sql : str SQL query or SQLAlchemy Selectable (select or text object) SQL query to be executed. con : SQLAlchemy connectable, str, or sqlite3 connection Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. Useful for SQL result sets. params : list, tuple or mapping, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype : Type name or dict of columns Data type for data or columns. E.g. np.float64 or {'a': np.float64, 'b': np.int32, 'c': 'Int64'}. .. versionadded:: 1.3.0 dtype_backend : {'numpy_nullable', 'pyarrow'}, default 'numpy_nullable' Back-end data type applied to the resultant :class:`DataFrame` (still experimental). Behaviour is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` (default). * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` DataFrame. .. versionadded:: 2.0 Returns ------- DataFrame or Iterator[DataFrame] See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql : Read SQL query or database table into a DataFrame. Notes ----- Any datetime values with time zone information parsed via the `parse_dates` parameter will be converted to UTC. Examples -------- >>> from sqlalchemy import create_engine # doctest: +SKIP >>> engine = create_engine("sqlite:///database.db") # doctest: +SKIP >>> with engine.connect() as conn, conn.begin(): # doctest: +SKIP ... data = pd.read_sql_table("data", conn) # doctest: +SKIP rh)rrrir8rrTrjN)rrrr read_query) rrrrirr8rrTrjrs r9rrsl &&&&&  . . . . 3    :$$ %#'%                      sA))A-0A- list[str]c dSrr rrrrirr8rqrrjrTs r9read_sqlr Cr;c dSrrrs r9rr,rr;c t||tjurd}|tjusJt|5} t | t r)| |||||||| cdddS | |} n#t$rd} YnwxYw| r(| |||||||cdddS| |||||||| cdddS#1swxYwYdS)ak Read SQL query or database table into a DataFrame. This function is a convenience wrapper around ``read_sql_table`` and ``read_sql_query`` (for backward compatibility). It will delegate to the specific function depending on the provided input. A SQL query will be routed to ``read_sql_query``, while a database table name will be routed to ``read_sql_table``. Note that the delegated function might have more specific notes about their functionality not listed here. Parameters ---------- sql : str or SQLAlchemy Selectable (select or text object) SQL query to be executed or a table name. con : ADBC Connection, SQLAlchemy connectable, str, or sqlite3 connection ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. The user is responsible for engine disposal and connection closure for the ADBC connection and SQLAlchemy connectable; str connections are closed automatically. See `here `_. index_col : str or list of str, optional, default: None Column(s) to set as index(MultiIndex). coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'}. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table (only used when reading a table). chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype_backend : {'numpy_nullable', 'pyarrow'}, default 'numpy_nullable' Back-end data type applied to the resultant :class:`DataFrame` (still experimental). Behaviour is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` (default). * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` DataFrame. .. versionadded:: 2.0 dtype : Type name or dict of columns Data type for data or columns. E.g. np.float64 or {'a': np.float64, 'b': np.int32, 'c': 'Int64'}. The argument is ignored if a table is passed instead of a query. .. versionadded:: 2.0.0 Returns ------- DataFrame or Iterator[DataFrame] See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql_query : Read SQL query into a DataFrame. Examples -------- Read data from SQL via either a SQL query or a SQL tablename. When using a SQLite database only SQL queries are accepted, providing only the SQL tablename will result in an error. >>> from sqlite3 import connect >>> conn = connect(':memory:') >>> df = pd.DataFrame(data=[[0, '10/11/12'], [1, '12/11/10']], ... columns=['int_column', 'date_column']) >>> df.to_sql(name='test_data', con=conn) 2 >>> pd.read_sql('SELECT int_column, date_column FROM test_data', conn) int_column date_column 0 0 10/11/12 1 1 12/11/10 >>> pd.read_sql('test_data', 'postgres:///db_name') # doctest:+SKIP Apply date parsing to columns through the ``parse_dates`` argument The ``parse_dates`` argument calls ``pd.to_datetime`` on the provided columns. Custom argument values for applying ``pd.to_datetime`` on a column are specified via a dictionary format: >>> pd.read_sql('SELECT int_column, date_column FROM test_data', ... conn, ... parse_dates={"date_column": {"format": "%d/%m/%y"}}) int_column date_column 0 0 2012-11-10 1 1 2010-11-12 .. versionadded:: 2.2.0 pandas now supports reading via ADBC drivers >>> from adbc_driver_postgresql import dbapi # doctest:+SKIP >>> with dbapi.connect('postgres:///db_name') as conn: # doctest:+SKIP ... pd.read_sql('SELECT int_column FROM test_data', conn) int_column 0 0 1 1 rh)rrrir8rrjrTNFr) rrrrrNSQLiteDatabaserr Exceptionr) rrrrirr8rqrrjrTr_is_table_names r9rr<sB &&&&&  . . . . 3  ': j. 1 1 ((#)'#+)  '''''''' #'11#66NN # # #"NNN #  ((#)'#+))''''''''<((#)'#+)  =''''''''''''''''''s;2C:?BC: B$!C:#B$$ C:C::C>C>failautoname if_exists$Literal['fail', 'replace', 'append']index index_labelIndexLabel | Nonemethod"Literal['multi'] | Callable | Nonerc T|dvrtd|dt|tr|}n$t|tst dt ||d5} | j||f||||||| | d| cdddS#1swxYwYdS) a Write records stored in a DataFrame to a SQL database. Parameters ---------- frame : DataFrame, Series name : str Name of SQL table. con : ADBC Connection, SQLAlchemy connectable, str, or sqlite3 connection or sqlite3 DBAPI2 connection ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. schema : str, optional Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). if_exists : {'fail', 'replace', 'append'}, default 'fail' - fail: If table exists, do nothing. - replace: If table exists, drop it, recreate it, and insert data. - append: If table exists, insert data. Create if does not exist. index : bool, default True Write DataFrame index as a column. index_label : str or sequence, optional Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. chunksize : int, optional Specify the number of rows in each batch to be written at a time. By default, all rows will be written at once. dtype : dict or scalar, optional Specifying the datatype for columns. If a dictionary is used, the keys should be the column names and the values should be the SQLAlchemy types or strings for the sqlite3 fallback mode. If a scalar is provided, it will be applied to all columns. method : {None, 'multi', callable}, optional Controls the SQL insertion clause used: - None : Uses standard SQL ``INSERT`` clause (one per row). - ``'multi'``: Pass multiple values in a single ``INSERT`` clause. - callable with signature ``(pd_table, conn, keys, data_iter) -> int | None``. Details and a sample callable implementation can be found in the section :ref:`insert method `. engine : {'auto', 'sqlalchemy'}, default 'auto' SQL engine library to use. If 'auto', then the option ``io.sql.engine`` is used. The default ``io.sql.engine`` behavior is 'sqlalchemy' .. versionadded:: 1.3.0 **engine_kwargs Any additional kwargs are passed to the engine. Returns ------- None or int Number of rows affected by to_sql. None is returned if the callable passed into ``method`` does not return an integer number of rows. .. versionadded:: 1.4.0 Notes ----- The returned rows affected is the sum of the ``rowcount`` attribute of ``sqlite3.Cursor`` or SQLAlchemy connectable. If using ADBC the returned rows are the result of ``Cursor.adbc_ingest``. The returned value may not reflect the exact number of written rows as stipulated in the `sqlite3 `__ or `SQLAlchemy `__ )rreplacerv'' is not valid for if_existsz9'frame' argument should be either a Series or a DataFrameTr)rrrrrrTrrN)rRrNr"to_framer!NotImplementedErrorrto_sql) rrrrrrrrrTrr engine_kwargsrs r9rrs3j555DYDDDEEE%      y ) ) ! G    3v E E E   z     #                         s8BB!$B!ct||5}||cdddS#1swxYwYdS)a Check if DataBase has named table. Parameters ---------- table_name: string Name of SQL table. con: ADBC Connection, SQLAlchemy connectable, str, or sqlite3 connection ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library. If a DBAPI2 object, only sqlite3 is supported. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). Returns ------- boolean rN)rr)rrrrs r9rr[s* 3v . . .0*##J//000000000000000000s 488r PandasSQLcddl}t||js|t|St dd}t|t r|t d|2t|t |jjfrt|||St dd}|r$t||jrt|Stj dtt t|S) z Convenience function to return the correct PandasSQL subclass based on the provided parameters. Also creates a sqlalchemy connection and transaction if necessary. rNrrBrz.Using URI string without sqlalchemy installed.zadbc_driver_manager.dbapizpandas only supports SQLAlchemy connectable (engine/connection) or database string URI or sqlite3 DBAPI2 connection. Other DBAPI2 objects are not tested. Please consider using SQLAlchemy.r)sqlite3rN Connectionrrr ImportErrorr Connectable SQLDatabase ADBCDatabaserr UserWarningr)rrrrradbcs r9rrwsNNN#w)**#ckc"""+LJJJJ#sL 2JKKK*S3 8I8U2V"W"W3(8999 %&A( S S SD ! 300!C    M D #%%  #  r;ceZdZdZ d4d5dZdZd6dZd7dZd7dZd8dZ d8dZ d9dZ d:d;d!Z dd?d+Zd,Zd-Zd.Z d@dAd/ZdBd2Zd3ZdS)CSQLTablez For mapping Pandas tables to SQL tables. Uses fact that table is reflected by SQLAlchemy to do better type conversions. Also holds various flags needed to avoid having to pass them between functions all the time. NTrpandasrrrbool | str | list[str] | NonerrprefixrTrrlrc ||_||_||_||_||||_||_||_| |_| |_ || |_ n*|j |j|j|_ |j td|dt|jstddS)NzCould not init table 'rzEmpty table name specified)rpd_sqlrr _index_namerrrkeysrT_create_table_setupr get_tablerRry) selfrpandas_sql_enginerrrrrrrrTs r9__init__zSQLTable.__init__s '   %%e[99  "   1133DJJ..ty$+FFDJ : =d===>> >49~~ ;9:: : ; ;r;cL|j|j|jSr)rrrrrs r9existszSQLTable.existss{$$TY <<>>$$&&&&&8++ !QT^!Q!Q!QRRR  " " " " "r;rrrcfd|D}||j|}|jS)a< Execute SQL statement inserting data Parameters ---------- conn : sqlalchemy.engine.Engine or sqlalchemy.engine.Connection keys : list of str Column names data_iter : generator of list Each item contains a list of values to be inserted cJg|]}tt| SrrOrw.0rowrs r9 z,SQLTable._execute_insert..):::Ss^^$$:::r;)rrinsertrowcount)rconnr data_iterrzresults ` r9_execute_insertzSQLTable._execute_insertsG;::: :::dj//11488r;cddlm}fd|D}||j|}||}|jS)a  Alternative to _execute_insert for DBs support multi-value INSERT. Note: multi-value insert is usually faster for analytics DBs and tables containing a few columns but performance degrades quickly with increase of columns. r)rcJg|]}tt| Srr rs r9rz2SQLTable._execute_insert_multi..rr;)rrrvaluesrr)rrrrrrzstmtrs ` r9_execute_insert_multizSQLTable._execute_insert_multish &%%%%%:::: :::vdj!!((..d##r;"tuple[list[str], list[np.ndarray]]c|jg|j}|j|j_ |dn,#t $r}t d||d}~wwxYw|j}t tt|j }t|}dg|z}t| D]\}\}}|j jdkrt|jt"rddl} | j|j jr"|jt.} n4t1j5t1jdt6t9j|jt.} dddn #1swxYwYn|j} n|j jd krr|j} t| t"r(| t9j d } | d !t.} n|j!t.} t| t8j"sJtG| |j$rtK| } d| | <| ||<||fS) NTinplacez!duplicate name in index/columns: Mr)rTrB)categoryrGzm8[ns]i8)&rrcopynames reset_indexrRrsmaprrqryr^r_rTkindrN_valuesr#rntypesis_date pyarrow_dtypeto_numpyobjectrcatch_warningsfilterwarningsrrVasarraydt to_pydatetimeviewrndarrayrU _can_hold_nar) rtemperr column_namesncols data_listrc_serr}rEvalsmasks r9 insert_datazSQLTable.insert_datas : !:??$$D#zDJ  U   .... U U U !JS!J!JKKQTT U:DCT\2233 L!!(,fun $TZZ\\22  KAx3y~$$ck+>?? 4((((x'' (?@@QK00v0>>%466QQ$3H}UUUU " 36+?+?+A+A P P PAQQQQQQQQQQQQQQQ  1133AA3&&{d$788C==rx/A/A=BBDIIdOO**622K&&v..a,, 5 5d1gg 5 5, Aww$IaLLY&&s*A A,A''A,#AF==G G rrrrcv ||j}n@|dkr|j}n2t|rt||}nt d||\}}t |j}|dkrdS||}n|dkrt d||zdz}d}|j 5} t|D]T} | |zt| dz|z|  krn0t fd|D} || || } | || }O|| z }Udddn #1swxYwY|S)NmultizInvalid parameter `method`: rz%chunksize argument should be non-zeroc3,K|]}|VdSrr)rrend_istart_is r9 z"SQLTable.insert..`s,"K"K#3wu}#5"K"K"K"K"K"Kr;) rrcallabler rRrAryrrrrxminrw)rrr exec_insertrr<nrowschunkstotal_insertedrrc chunk_iter num_insertedrFrGs @@r9rzSQLTable.insert<s >.KK w  4KK f   F!&$//KKDFDDEE E**,,iDJ A::1  II !^^DEE E9$) [ ( ( * * 7d6]] 7 7i-QUi/77e##E "K"K"K"K"K"K"K"KL *{4zBB +%-)5&,6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7s=A%D..D25D2rh exit_stackrrir=rjrkc#rKd}|5 ||} | s|stjg||Vnbd}t| ||||_||||j!|j|jd|jV ddddS#1swxYwYdS)z,Return generator through chunked result set.FTrqrir8rjNr ) fetchmanyr! from_recordsrr_harmonize_columnsrr) rrrQrrqrir8rj has_read_datarzs r9_query_iteratorzSQLTable._query_iteratorjsQ  ! ! !'' 22('4l $ 9'< '' +=(:)J((T(BBBj   + ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !sBB,,B03B0rc |ddlm}|gt|dkrTfd|D}j9jdddD](} |djj| )||} n|j} j| } | } | | ||| |||S| } t| | ||_ ||j!j jdj S) Nr)selectc4g|]}jj|Sr)rc)rnrs r9rz!SQLTable.read..s!555DJLO555r;)rir8rjrTTr )rr[ryrrrr]rrrrYfetchallrrrWr)rrQrir8rqrrjr[colsidx sql_selectrr:rzs` r9readz SQLTable.reads &%%%%%  3w<>>: r;c|dur|jjj}|?t|ts|g}t ||krt d||S|dkr"d|jjvr|jjjdgStj |jjj St|tr|gSt|tr|SdS)NTz@Length of 'index_label' should match number of levels, which is rDr) rrnlevelsrNrsryrRrqrcomfill_missing_namesr&r)rrrrfs r9rzSQLTable._index_names D==j&.G&!+t440#.-K{##w..$6,366#"1 4:#555J$)1y -dj.>.DEEEs # # 7N t $ $ L4r;cVg}jgtjD]R\}}jj|}|t ||dfS|fdt tjjDz }|S)NTcg|]A}tjj|jjdd|fdfBSNF)rrrqiloc)rrc dtype_mapperrs r9rz8SQLTable._get_column_names_and_types..s]# # # #A& ' 'djoaaad6K)L)Le T# # # r;) rr^r_get_level_valuesrvrrxryrq)rrmcolumn_names_and_typesrc idx_labelidx_types`` r9_get_column_names_and_typesz$SQLTable._get_column_names_and_typess!# : ! )$* 5 5 P P 9'< (8(J(J1(M(MNN&--s9~~x.NOOOO# # # # # 3tz12233# # #   &%r;c| ddlm m}m}ddlm}||j} fd|D}|jIt|js |jg}n|j}||d|j dzi}| ||j p|j jj }|} ||j | g|Rd|iS)Nr)ColumnPrimaryKeyConstraintr,MetaDatac4g|]\}}}|||S))rr)rrtypis_indexrts r9rz0SQLTable._create_table_setup..s@   #c8 F4H - - -   r;r_pkr)rrtrur,rrwrr_sqlalchemy_typerrrrvrrr) rrur,rwrorqrpkcrrrts @r9rzSQLTable._create_table_setups)          /.....!%!A!A$BW!X!X    '=    9  ** ! {y&&E49u3DEEC NN3   7 0 7xzzuTY>w>>>v>>>r;ct|}|jjD]}|j} |j|}||vr6 ||}n#t $rd}YnwxYwt |||j|<R||j}|tus|tus |tur#|tu}t |||j|<n|dkr)|tur | |d|j|<ntrIt|r:t!|j|r | |d|j|<ni|dkrct#||kr>|t'jdus |t*ur| |d|j|<#t,$rYwxYwdS)a Make the DataFrame's column types align with the SQL table column types. Need to work around limited NA value support. Floats are always fine, ints must always be floats if there are Null values. Booleans are hard because converting bool column with None replaces all Nones with false. Therefore only convert bool if there are no NA values. Datetimes should already be converted to np.datetime64 if supported, but here we also force conversion if required. Nr]rMrhF)r%int64)r:rrqrrrQr[ _get_dtyperUr rrfloatrrrrrycountrVrTr=r`) rr8rjsql_colrdrerfcol_typer<s r9rWzSQLTable._harmonize_columns s 4K@@ z)( ( G|H& H-{**#)(3$###"#+>vc+R+R+RDJx( ??7<88((4''?22#o5C+>v3+O+O+ODJx(("g--(e2C2C+1===+N+NDJx((&(( S'11 S( 8(<== S ,2===+N+NDJx(("g--#f++2O2O28G#4#444D8H8H/5}}XE}/R/R 8,    O( ( s;F7AF7 AF7AF7.EF77 GGrYIndex | Seriesc|jpi}t|r+tt|}|j|vr ||jSt j|d}ddlm}m }m }m }m }m } m} m} m} m} |dvrK |jj |dSn/#t($r"t+|dd|dcYSYnwxYw|S|dkr*t-jd t0t3 |S|d kr#|jd kr | d S| dS|dkrw|jjdvr| S|jjdvr| S|jjdkrt7d|S|dkr|S|dkr|S|dkr| S|dkrt7d| S)NTskipnar) TIMESTAMP BigIntegerBooleanDateDateTimeFloatInteger SmallIntegerTextTime) datetime64r )timezonetz timedelta64lthe 'timedelta' type is not supported, and will be written as integer values (ns frequency) to the database.rrWfloat32) precision5rX)int8uint8int16)uint16int32uint64z1Unsigned 64 bit integer datatype is not supportedbooleanrr complexComplex datatypes not supported)rTrrrOrr infer_dtypesqlalchemy.typesrrrrrrrrrrr3rAttributeErrorgetattrrrrrlowerrR)rrYrTrrrrrrrrrrrs r9r|zSQLTable._sqlalchemy_typeEs**    'u%%Ex5  SX&?3t444  1 1 1 469($9d3333)! 4 4 43d++7$9d33333387 4 O } $ $ ML+--         # #yI%%ur****ur****  " "y~##%%)CCC##%%''+>>>%%''833 !TUUU!!  " "N   K   K  " ">?? ? s7B'B<;B<cddlm}m}m}m}m}m}m}t||rtSt||rtj dSt||r|j stStSt||rtSt||rtSt||rt St||r(t#rt%tjSt(S)Nr)rrrrrrStringr)na_value)rrrrrrrrrNrrVrTrr rrr=rr$nanr/) rsqltyperrrrrrrs r9rzSQLTable._get_dtypes<                   gu % % 4L  ) ) 48G$$ $  + + 4# " "  * * 4O  & & 4K  ) ) 4K  ( ( 4!## 4"BF3333 r;)NTrrNNNN) rrrrrrrrrTrrlrrlrrlr)rrrlr)rlr)NN)rrrrrlr)TNrh)rQrrrrir=rjrk)TNNNrh) rQrrir=rrrjrkrlr)Nrh)rjrkrlr)rYr)__name__ __module__ __qualname____doc__rrrrrrrrArrYrdrrrrrWr|rrr;r9rrs/3:@ !%";";";";";H===EEEE 4444 # # # # "2'2'2'2'l!%59,,,,,h"9@"!"!"!"!"!N" $9@-----^@ & & &???@9@:::::xEEEEN     r;rceZdZdZd.dZd/dZ d0d1dZe d0d2dZe d3d4d%Z ed5d6d'Z ed5d7d(Z e d8d9d-Z dS):rz9 Subclasses Should define read_query and to_sql. rlr3c|Srrrs r9 __enter__zPandasSQL.__enter__s r;rcdSrrrargss r9__exit__zPandasSQL.__exit__s r;NTrhrrrrrir=rrrrrjrkrc trr) rrrrir8rqrrrjs r9rzPandasSQL.read_tables "!r;rrTrc dSrr) rrrrir8rrrTrjs r9rzPandasSQL.read_querys  r;rrrrrrrrrc  dSrr) rrrrrrrrrTrrrs r9rzPandasSQL.to_sqls  r;str | Select | TextClausecdSrr)rrrs r9rzPandasSQL.execute r;cdSrr)rrrs r9rzPandasSQL.has_tablerr;rr!rrcdSrrrrrrrTrs r9_create_sql_schemazPandasSQL._create_sql_schemas  r;)rlr3rNTNNNNrhrrrrrir=rrrrrjrkrlrrrrrrir=rrrTrrjrkrlrrTNNNNNr)rrrrrr=rrrTrrrrrrlrrrrrrrrrlr=NNN rr!rrrrrTrrrrlr) rrrrrrrrrrrrrrr;r9rrsl    -1!! $9@ " " " " "-1! $!%9@     ^   ;A $!%59    ^     ^     ^  "&!%!     ^   r;c eZdZ d dd ZdS) BaseEngineTNrrrrrrrrrlc   t|)z: Inserts data into already-prepared table )r) rrrrrrrrrrs r9insert_recordszBaseEngine.insert_recordss"$'''r;TNNN rrrrrrrrrlr)rrrrrr;r9rrs;04 $(((((((r;rc(eZdZddZ dddZdS)SQLAlchemyEnginerlrc(tdddS)Nrz'sqlalchemy is required for SQL support.)extrarrs r9rzSQLAlchemyEngine.__init__s)"  I      r;TNrrrrrrrrc  ddlm}  |||S#| j$rB} d} t | j} t j| | rtd| | d} ~ wwxYw)Nr)exc)rrzg(\(1054, "Unknown column 'inf(e0)?' in 'field list'"\))(?# )|inf can not be used with MySQLzinf cannot be used with MySQL) rrrStatementErrorrorigresearchrR)rrrrrrrrrrrr9msgerr_texts r9rzSQLAlchemyEngine.insert_recordss #""""" <<)F<CC C!   0C38}}Hyh'' K !@AAsJI s A+=A&&A+rrr)rrrrrrr;r9rr sO    04 $r;rc0|dkrtd}|dkrYtg}d}|D]:} |cS#t$r}|dt|zz }Yd}~3d}~wwxYwtd||dkrtSt d) zreturn our implementationrz io.sql.enginez - NzUnable to find a usable engine; tried using: 'sqlalchemy'. A suitable version of sqlalchemy is required for sql I/O support. Trying to import the above resulted in these errors:rz*engine must be one of 'auto', 'sqlalchemy')r rrrrR)rengine_classes error_msgs engine_classr9s r9 get_enginer.s O,, *+ * 1 1L 1#|~~%%% 1 1 1gC00  1        !!! A B BBs 7 A AA ceZdZdZ dd Zed Zd?d@dZ dAdBdZ e dCdDd"Z dAdEd#Z e Z dFdGd+ZdHd,Z dIdJd1Zed2Zd?dKd3Zd?dLd5Zd?dMd6Z dNdOd;ZdS)Praa This class enables conversion between DataFrame and SQL databases using SQLAlchemy to handle DataBase abstraction. Parameters ---------- con : SQLAlchemy Connectable or URI string. Connectable to connect with the database. Using SQLAlchemy makes it possible to use any DB supported by that library. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If None, use default schema (default). need_transaction : bool, default False If True, SQLDatabase will create a transaction. NFrrrr=rlrcddlm}ddlm}ddlm}t |_t|tr*||}|j |j t||r,|j | }|r@|s,|j |||_|||_d|_dS)Nr) create_engine)rrvrF)rrsqlalchemy.enginerrrwrrQrNrcallbackdispose enter_contextconnectin_transactionbeginrrreturns_generator)rrrrrrrws r9rzSQLDatabase.__init__`s -,,,,,,,,,,,...... $++ c3   2-$$C O $ $S[ 1 1 1 c6 " " ?/// >>C  7C$6$6$8$8 7 O ) )#))++ 6 6 6HF+++ !&r;cJ|js|jdSdSr)rrQclosers r9rzSQLDatabase.__exit__ws1% $ O ! ! # # # # # $ $r;c#K|js=|j5|jVddddS#1swxYwYdS|jVdSr)rrrrs r9rzSQLDatabase.run_transaction{sx&&(( !!  h                  (NNNNNs A  AArrc|gn|g}t|tr|jj|g|RS|jj|g|RS)z,Simple passthrough to SQLAlchemy connectable)rNrrexec_driver_sqlr)rrrrs r9rzSQLDatabase.executes]^rr& c3   8+48+C7$777 7tx+d++++r;Trhrrrrrirrrjrkrc |j|j|gdt||||} |d|_| |j|||||S)ap Read SQL database table into a DataFrame. Parameters ---------- table_name : str Name of SQL table in database. index_col : string, optional, default: None Column to set as index. coerce_float : bool, default True Attempts to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point. This can result in loss of precision. parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg}``, where the arg corresponds to the keyword arguments of :func:`pandas.to_datetime`. Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table. schema : string, default None Name of SQL schema in database to query (if database flavor supports this). If specified, this overwrites the default schema of the SQL database object. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype_backend : {'numpy_nullable', 'pyarrow'}, default 'numpy_nullable' Back-end data type applied to the resultant :class:`DataFrame` (still experimental). Behaviour is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` (default). * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` DataFrame. .. versionadded:: 2.0 Returns ------- DataFrame See Also -------- pandas.read_sql_table SQLDatabase.read_query T)ronlyviews)rrN)rir8rqrrj)rreflectrrrrdrQ) rrrrir8rqrrrjrs r9rzSQLDatabase.read_tablesy~ txzl$GGGT6JJJ  %)D "zz O%#'    r;rQrrrTrc #Kd} |5 ||} | s| stg||||||Vnd} t| ||||||VM ddddS#1swxYwYdS)+Return generator through chunked result setFTrrir8rTrjN)rUr) rrQrrqrrir8rTrjrXrzs r9rYzSQLDatabase._query_iterators    '' 22 ( *#&/)5(3"'*7 $ "'!- +"/!                   sAA$$A(+A(c |||} | } |*d|_|| |j|| ||||| S| } t | | |||||} | S)a Read SQL query into a DataFrame. Parameters ---------- sql : str SQL query to be executed. index_col : string, optional, default: None Column name to use as index for the returned DataFrame object. coerce_float : bool, default True Attempt to convert values of non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets. params : list, tuple or dict, optional, default: None List of parameters to pass to execute method. The syntax used to pass parameters is database driver dependent. Check your database driver documentation for which of the five syntax styles, described in PEP 249's paramstyle, is supported. Eg. for psycopg2, uses %(name)s so use params={'name' : 'value'} parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg dict}``, where the arg dict corresponds to the keyword arguments of :func:`pandas.to_datetime` Especially useful with databases without native Datetime support, such as SQLite. chunksize : int, default None If specified, return an iterator where `chunksize` is the number of rows to include in each chunk. dtype : Type name or dict of columns Data type for data or columns. E.g. np.float64 or {'a': np.float64, 'b': np.int32, 'c': 'Int64'} .. versionadded:: 1.3.0 Returns ------- DataFrame See Also -------- read_sql_table : Read SQL database table into a DataFrame. read_sql NTr)rrrrYrQr`r) rrrrir8rrrTrjrrqrzrs r9rzSQLDatabase.read_querysrc6**++--  %)D "''#)'+(   ??$$D #)'+ELr;rrrrrrrc rtsfd|Dnttddlm}D]N\} } t | trt| |r+t | |rz*SQLDatabase.prep_table..l???X5???r;r) TypeEnginez The type of z is not a SQLAlchemy type)rrrrrrT) rrrOrrr_rNrUrSrRrr) rrrrrrrrTrrYmy_typers ` r9 prep_tablezSQLDatabase.prep_tableWs  T&& *@??????T5)) 3 3 3 3 3 3 %  T T Wgt,,TGZ1P1PT44T$%RC%R%R%RSSS  #       r;c6|s|snddlm}||j}||p |jj}||vr4d|d}tj |ttdSdSdSdS)zv Checks table name for issues with case-sensitivity. Method is called after data is inserted. rinspectrzThe provided table name 'z' is not found exactly as such in the database after writing the table, possibly due to case sensitivity issues. Consider using lower case table names.rN) isdigitislowerrr rget_table_namesrrrrrr)rrrsqlalchemy_inspectinsp table_namesrs r9check_case_sensitivez SQLDatabase.check_case_sensitives||~~ dllnn  A @ @ @ @ @%%dh//D..f6P @P.QQK;&&((((  /11    '&r;rrrrc t| } ||||||||} | jd| |j|||||| d| }||||S)a Write records stored in a DataFrame to a SQL database. Parameters ---------- frame : DataFrame name : string Name of SQL table. if_exists : {'fail', 'replace', 'append'}, default 'fail' - fail: If table exists, do nothing. - replace: If table exists, drop it, recreate it, and insert data. - append: If table exists, insert data. Create if does not exist. index : boolean, default True Write DataFrame index as a column. index_label : string or sequence, default None Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If specified, this overwrites the default schema of the SQLDatabase object. chunksize : int, default None If not None, then rows will be written in batches of this size at a time. If None, all rows will be written at once. dtype : single type or dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a SQLAlchemy type. If all columns are of the same type, one single value can be used. method : {None', 'multi', callable}, default None Controls the SQL insertion clause used: * None : Uses standard SQL ``INSERT`` clause (one per row). * 'multi': Pass multiple values in a single ``INSERT`` clause. * callable with signature ``(pd_table, conn, keys, data_iter)``. Details and a sample callable implementation can be found in the section :ref:`insert method `. engine : {'auto', 'sqlalchemy'}, default 'auto' SQL engine library to use. If 'auto', then the option ``io.sql.engine`` is used. The default ``io.sql.engine`` behavior is 'sqlalchemy' .. versionadded:: 1.3.0 **engine_kwargs Any additional kwargs are passed to the engine. )rrrrrrrT)rrrrrrrr)rrr)rrrrr)rrrrrrrrrTrrr sql_enginerrNs r9rzSQLDatabase.to_sqls| '' #   32         !!tF!;;;r;c|jjSr)rtablesrs r9rzSQLDatabase.tabless yr;crddlm}||j}|||p |jjS)Nrr)rr rrrr)rrrr rs r9rzSQLDatabase.has_tablesD<<<<<<!!$(++~~dF$>di.>???r;r,cddlm}m}|p |jj}|||j|j|}|jD]#}t|j|r d|j_ $|S)Nr)Numericr,) autoload_withrF) rrr,rrrrqrNrU asdecimal)rrrrr,tblcolumns r9rzSQLDatabase.get_tables        +49+eJ &QQQk . .F&+w// .(- % r;c|p |jj}|||r|j|j|g|d|5||||jdddn #1swxYwY|jdSdS)NT)rrrrr) rrrrrrrdropclearrrrs r9r zSQLDatabase.drop_tables+49+ >>*f - -  I  XZLt    %%'' G Gz62277TX7FFF G G G G G G G G G G G G G G G IOO        s0BB Brr!rrc pt|||d|||}t|SNF)rrrrTr)rrrrrrrrTrrs r9rzSQLDatabase._create_sql_schemasI     5##%%&&&r;rk)rrrr=rlrrrrrrNTNNrh) rQrrrrir=rTrrjrkr)rTNNN) rrrrrrrTrrlrrrrrrlrrrrrrrr=rrrrrTrrrrrrlrr)rrrrrlr,rrrrrlrrr)rrrrrrrrrr staticmethodrYrrrrrpropertyrrrr rrr;r9rrNs:$HM'''''.$$$$^,,,,,-1!! $9@J J J J J X !!%9@&&&&\&V-1! $!%9@TTTTTlH ;A/3!%.....`B;A! $!%59WWWWWr  X @@@@@     "&!%! '''''''r;rceZdZdZd-dZedZd.d/d Z d0d1dZ d0d2dZ e Z d3d4d&Z d.d5d'Z d6d7d,Z dS)8rz This class enables conversion between DataFrame and SQL databases using ADBC to handle DataBase abstraction. Parameters ---------- con : adbc_driver_manager.dbapi.Connection rlrc||_dSrrrrs r9rzADBCDatabase.__init__? r;c#K|j5} |Vn(#t$r|jwxYw|jddddS#1swxYwYdSr)rcursorrrollbackcommitrcurs r9rzADBCDatabase.run_transactionBs X__   #     !!###  HOO                      s%A1#A1%AA11A58A5Nrrc t|tstd|gn|g}|j} |j|g|R|S#t $rc} |jn+#t $r}td|d|d}||d}~wwxYwtd|d|}||d}~wwxYwNz/Query must be a string unless using sqlalchemy.zExecution failed on sql:  z unable to rollbackzExecution failed on sql 'z': rNrrQrr/rrr0rrrrrr3r inner_excexs r9rzADBCDatabase.executeL#s## OMNN N^rr&hoo  CK #d # # # #J    (!!#### ( ( ("PPPsPPPi'  ( H3HH3HHIIB#  6A C!A;:B>; B#BB##B>>CTrhrrrrrir=rrrrrjrkrc |durtd|rtd|r9|rt|} ng} | |z} dd| D} nd} |r d| d|d |} nd| d|} |j5} | | | }t|| }d d d n #1swxYwYt||| S) a9 Read SQL database table into a DataFrame. Parameters ---------- table_name : str Name of SQL table in database. coerce_float : bool, default True Raises NotImplementedError parse_dates : list or dict, default: None - List of column names to parse as dates. - Dict of ``{column_name: format string}`` where format string is strftime compatible in case of parsing string times, or is one of (D, s, ns, ms, us) in case of parsing integer timestamps. - Dict of ``{column_name: arg}``, where the arg corresponds to the keyword arguments of :func:`pandas.to_datetime`. Especially useful with databases without native Datetime support, such as SQLite. columns : list, default: None List of column names to select from SQL table. schema : string, default None Name of SQL schema in database to query (if database flavor supports this). If specified, this overwrites the default schema of the SQL database object. chunksize : int, default None Raises NotImplementedError dtype_backend : {'numpy_nullable', 'pyarrow'}, default 'numpy_nullable' Back-end data type applied to the resultant :class:`DataFrame` (still experimental). Behaviour is as follows: * ``"numpy_nullable"``: returns nullable-dtype-backed :class:`DataFrame` (default). * ``"pyarrow"``: returns pyarrow-backed nullable :class:`ArrowDtype` DataFrame. .. versionadded:: 2.0 Returns ------- DataFrame See Also -------- pandas.read_sql_table SQLDatabase.read_query T2'coerce_float' is not implemented for ADBC drivers/'chunksize' is not implemented for ADBC drivers, c3"K|] }d|dV dS)"Nr)rxs r9rHz*ADBCDatabase.read_table..s*#@#@HHHH#@#@#@#@#@#@r;*zSELECT z FROM .rjN)rr8) rr&joinrr/rfetch_arrow_tabler)r)rrrrir8rqrrrj index_select to_select select_listrr3pa_tablers r9rzADBCDatabase.read_table`st t # #%D   Y%&WXX X   ".y99 ! $w.I))#@#@i#@#@#@@@KKK  =E[EEEEEEDD<[<< <z,'params' is not implemented for ADBC driversr?rFN)rr8rT)rrr/rrHr)r) rrrrir8rrrTrjr3rLrs r9rzADBCDatabase.read_querys%f t # #%D   V%&TUU U  Y%&WXX X X__   N# KK   ,,..H&x}MMMB N N N N N N N N N N N N N N N ! #     s;BBBrrrrrrrrrc  0|rtd|rtd|rtd| rtd| dkrtd|r|d|} n|} d} |||rq|d krtd | d |d krJ|j5}|d | dddn #1swxYwYn|dkrd} ddl} |j||}n"#|j $r}td|d}~wwxYw|j5}| ||| |}dddn #1swxYwY|j |S)a Write records stored in a DataFrame to a SQL database. Parameters ---------- frame : DataFrame name : string Name of SQL table. if_exists : {'fail', 'replace', 'append'}, default 'fail' - fail: If table exists, do nothing. - replace: If table exists, drop it, recreate it, and insert data. - append: If table exists, insert data. Create if does not exist. index : boolean, default True Write DataFrame index as a column. index_label : string or sequence, default None Raises NotImplementedError schema : string, default None Name of SQL schema in database to write to (if database flavor supports this). If specified, this overwrites the default schema of the SQLDatabase object. chunksize : int, default None Raises NotImplementedError dtype : single type or dict of column name to SQL type, default None Raises NotImplementedError method : {None', 'multi', callable}, default None Raises NotImplementedError engine : {'auto', 'sqlalchemy'}, default 'auto' Raises NotImplementedError if not set to 'auto' z1'index_label' is not implemented for ADBC driversr?z+'dtype' is not implemented for ADBC driversz,'method' is not implemented for ADBC driversrz1engine != 'auto' not implemented for ADBC driversrErrrr r DROP TABLE Nrvr)preserve_indexzdatatypes not supported)rrzmodedb_schema_name) rrrRrr/rrnr,roArrowNotImplementedError adbc_ingestr1)rrrrrrrrrTrrrrrQr3r}rrrNs r9rzADBCDatabase.to_sql sV  %C   Y%&WXX X  U%&STT T  V%&TUU U V  %C   "++T++JJJ  >>$ ' ' F"" !H:!H!H!HIIIi''X__&&<#KK :j : :;;;<<<<<<<<<<<<<<<h&& A(&&uU&CCCC* A A A677S @ AX__   # __cV-N                s<6CC"C4D D0D++D0 E22E69E6c|j||}|dD])}|s|D]!}|s|dD]}|d|krdS"*dS)N)db_schema_filtertable_name_filtercatalog_db_schemasdb_schema_tablesrTF)radbc_get_objectsread_all to_pylist)rrrrcatalog_schema schema_record table_records r9rzADBCDatabase.has_table` sx((#t)  (** ##78BBDD $ $N! !/ $ $ $$12D$E$$L#L1T99#tttt:$  $ur;rr!rrc td)Nznot implemented for adbcrrs r9rzADBCDatabase._create_sql_schemar s""<===r;rrrrrrrr%rrr)rrrrrrrrrrrrrrrr;r9rr5sP^.-1!! $9@X X X X X z-1! $!%9@F F F F F PH ;A! $!%59ZZZZZx,"&!%! >>>>>>>r;rTEXTREALINTEGERrDATETIME)rprWrXr rr rr/c t|ddd}n&#t$r}t d|d|d}~wwxYw|S)Nzutf-8strictz%Cannot convert identifier to UTF-8: 'r)rencodedecode UnicodeErrorrR)runamer9s r9_get_unicode_namerl swSD   (33::7CC SSSHHHHIIsRS Ls69 AAAct|}t|std|d}|dkrtdd|ddzdzS)Nz$Empty table or column name specifiedrz%SQLite identifier cannot contain NULsrBz"")rlryrRfindr)rrk nul_indexs r9_get_valid_sqlite_namerq sv d # #E u::A?@@@ 6""IA~~@AAA sD)) )C //r;c`eZdZdZdfd ZddZddZddZdd Zdd Z dd Z dZ dZ xZ S) SQLiteTablezw Patch the SQLTable for fallback support. Instead of a table variable just use the Create Table statement. rlrcbtj|i||dSr)superr_register_date_adapters)rrkwargs __class__s r9rzSQLiteTable.__init__ s7$)&))) $$&&&&&r;c(ddl}d d}d}d}|t||t||t|d}d}|d ||d |dS) NrrlrcL|jdd|jdd|jdd|jdS)N02d:rE06d)hourminutesecond microsecond)ts r9 _adapt_timez8SQLiteTable._register_date_adapters.._adapt_time s:fTTT18TTT!(TTTTTT Tr;c*|Sr isoformatvals r9z5SQLiteTable._register_date_adapters.. sS]]__r;c,|dS)N rrs r9rz5SQLiteTable._register_date_adapters.. ss););r;cNtj|Sr)r fromisoformatrirs r9rz5SQLiteTable._register_date_adapters.. s4#5cjjll#C#Cr;cNtj|Sr)r rrirs r9rz5SQLiteTable._register_date_adapters.. s(>szz||(L(Lr;r timestampr)rregister_adapterr rr register_converter)rrradapt_date_isoadapt_datetime_iso convert_dateconvert_timestamps r9rvz#SQLiteTable._register_date_adapters s  U U U U54;;  {333  ~666  +=>>>CC LL""6<888"";0ABBBBBr;rcPtd|jS)Nz; )rrGrrs r9rzSQLiteTable.sql_schema s5::dj))***r;c|j5}|jD]}|| ddddS#1swxYwYdSr)rrrr)rrrs r9rzSQLiteTable._execute_create s [ ( ( * * #d  # # T"""" # # # # # # # # # # # # # # # # # # #s AA A num_rowsrc ttt|jj}d}t |j)|jdddD]}|d| fd|D}d|}d|gt|z d fdt|D}d |j d|d |}|S) N?r_rc&g|] }|Srr)rrescapes r9rz0SQLiteTable.insert_statement.. s!>>>f66&>>>>>r;,cg|]}dd S)()r)rr= row_wildcardss r9rz0SQLiteTable.insert_statement.. s%LLLq2-222LLLr;z INSERT INTO  (z ) VALUES ) rsr(rrrqrqrrrGryrxr) rrr&wldrbbracketed_names col_names wildcardsinsert_statementrrs @@r9rzSQLiteTable.insert_statement s Sdj01122' : !z$$B$' % % Q$$$$>>>>>>>HH_-- #U!344 HHLLLLE(OOLLLMM O66$),, O O O OI O O  r;ct|}||d||jS)NrDr)rs executemanyrr)rrrrr<s r9rzSQLiteTable._execute_insert s<OO  ...::IFFF}r;ct|}d|D}||t|||jS)Ncg|] }|D]}| Srr)rrrCs r9rz5SQLiteTable._execute_insert_multi.. s%>>>#>>Q!>>>>r;r)rsrrryr)rrrrr<flattened_datas r9rz!SQLiteTable._execute_insert_multi sSOO >>y>>> T**C NN*CC^TTT}r;c0 ||j}t fd|D}|jzt |jrft |js |jg}n|j}d fd|D}|d|jd|d|j r |j dz}nd }d |z |jzd zd |zd zg}d|D}t |rd|}d fd|D}|d d|jzdz|zzdz |jzdz|zdz|S)z Return a list of SQL statements that creates a table reflecting the structure of a DataFrame. The first entry will be a CREATE TABLE statement while the rest will be CREATE INDEX statements. c:g|]\}}}|dz|zS)rr)rcnamectyper=rs r9rz3SQLiteTable._create_table_setup.. s>   ,;E5!FF5MMC % '   r;Nr@c&g|] }|Srrrr]rs r9rz3SQLiteTable._create_table_setup.. s!";";";66!99";";";r;z CONSTRAINT z_pk PRIMARY KEY (rrErz CREATE TABLE z ( z, z )cg|] \}}}|| Srr)rrr=rzs r9rz3SQLiteTable._create_table_setup.. s$VVV/UAxXV5VVVr;r=rc&g|] }|Srrrs r9rz3SQLiteTable._create_table_setup.. s!!=!=!=&&))!=!=!=r;z CREATE INDEX ix_zON r) rr_sql_type_namerqrryrrGrvrr) rrocreate_tbl_stmtsr cnames_br schema_name create_stmtsix_colscnamesrs @r9rzSQLiteTable._create_table_setup s) "&!A!A$BU!V!V'    ?U    9 S^^  ** ! {y ";";";";d";";";<>I   &*S069::;&##$        r;c|jpi}t|r+tt|}|j|vr ||jSt j|d}|dkr+tjdttd}n'|dkrd}n|d krd }n|d krtd |tvrd }t|S) NTrrrrrXrr emptyrprr) rTrrrOrrrrrrrrR _SQL_TYPES)rrYrTrs r9rzSQLiteTable._sql_type_name% s**    'u%%Ex5  SX&?3t444 } $ $ ML+--      !HH  % %!HH  HH  " ">?? ? : % %H(##r;rr)rrrlr)rlr)rrrrrrvrrrrrrr __classcell__)rxs@r9rsrs s '''''' CCCC:++++####     &  111f $ $ $ $ $ $ $r;rsceZdZdZd*dZedZd+d,d Ze d-d.dZ d/d0dZ dZ d1d2d"Z d+d3d%Z d+d4d'Zd+d5d(Z d6d7d)ZdS)8rz Version of SQLDatabase to support SQLite connections (fallback without SQLAlchemy). This should only be used internally. Parameters ---------- con : sqlite connection object rlrc||_dSrr+r,s r9rzSQLiteDatabase.__init__S r-r;c#&K|j} |V|jn(#t$r|jwxYw |dS#|wxYwr)rr/r1rr0rr2s r9rzSQLiteDatabase.run_transactionV shoo III HOO        H         IIKKKKKCIIKKKKs;A:%A  A::BNrrc t|tstd|gn|g}|j} |j|g|R|S#t $rc} |jn+#t $r}td|d|d}||d}~wwxYwtd|d|}||d}~wwxYwr5r7r8s r9rzSQLiteDatabase.executeb r;r<Trhrrrir=rTrrjrkc #RKd} ||} t| tkrt| } | sJ||s2t jg||} |r| |} | VdSd}t| ||||||V)rFTrSrN) rUrUtuplersrr!rVrr) r/rrqrrir8rTrjrXrzrs r9rYzSQLiteDatabase._query_iteratorv s  ##I..DDzzU""Dzz  $!&3G,F6!'u!5!5 LLL M#)'+   ! r;rrc |||} d| jD} ||| || |||||S|| } | t | | |||||} | S)Ncg|] }|d S)rr)rcol_descs r9rz-SQLiteDatabase.read_query.. sBBB88A;BBBr;r)r descriptionrY_fetchall_as_listrr) rrrrir8rrrTrjr/rqrzrs r9rzSQLiteDatabase.read_query sc6**BBv/ABBB  ''#)'+(   ))&11D LLNNN #)'+ELr;cv|}t|tst|}|Sr)r`rNrs)rr3rs r9rz SQLiteDatabase._fetchall_as_list s1&$'' "&\\F r;rrrrrrrrrc vrwtsfd|DnttD]/\} } t | t st | d| d0t||||||}|| || S)a@ Write records stored in a DataFrame to a SQL database. Parameters ---------- frame: DataFrame name: string Name of SQL table. if_exists: {'fail', 'replace', 'append'}, default 'fail' fail: If table exists, do nothing. replace: If table exists, drop it, recreate it, and insert data. append: If table exists, insert data. Create if it does not exist. index : bool, default True Write DataFrame index as a column index_label : string or sequence, default None Column label for index column(s). If None is given (default) and `index` is True, then the index names are used. A sequence should be given if the DataFrame uses MultiIndex. schema : string, default None Ignored parameter included for compatibility with SQLAlchemy version of ``to_sql``. chunksize : int, default None If not None, then rows will be written in batches of this size at a time. If None, all rows will be written at once. dtype : single type or dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a string. If all columns are of the same type, one single value can be used. method : {None, 'multi', callable}, default None Controls the SQL insertion clause used: * None : Uses standard SQL ``INSERT`` clause (one per row). * 'multi': Pass multiple values in a single ``INSERT`` clause. * callable with signature ``(pd_table, conn, keys, data_iter)``. Details and a sample callable implementation can be found in the section :ref:`insert method `. ci|]}|Srrrs r9rz)SQLiteDatabase.to_sql.. rr;rz) not a string)rrrrrT) rrrOr_rNrrRrsrr)rrrrrrrrrTrrrrYrrs ` r9rzSQLiteDatabase.to_sql sh  H&& *@??????T5)) %  H H W!'3//H$%F%Fw%F%F%FGGGH  #     ||Iv...r;rrcd}d|d}t|||gdkS)Nrz SELECT name FROM sqlite_master WHERE type IN ('table', 'view') AND name=z ; r)ryrr`)rrrrquerys r9rzSQLiteDatabase.has_table sS    4<<v..7799::Q>>r;rcdSrrrs r9rzSQLiteDatabase.get_table) str;cTdt|}||dS)NrO)rqr)rrrdrop_sqls r9r zSQLiteDatabase.drop_table, s0?!7!=!=?? Xr;c pt|||d|||}t|Sr!)rsrrr"s r9rz!SQLiteDatabase._create_sql_schema0 sI     5##%%&&&r;rrrr#)rrrir=rTrrjrkr) rir=rrrTrrjrkrlrr)rrrrrr=rrrTrrrrrrlrrr&r$r)rrrTrrrrlr)rrrrrrrrr'rYrrrrrr rrr;r9rrH s  ^ ( !!%9@$$$$\$R! $!%9@&&&&&P  $!%59N/N/N/N/N/` ? ? ? ? ?!%! '''''''r;rct|5}||||||cdddS#1swxYwYdS)a] Get the SQL db table schema for the given frame. Parameters ---------- frame : DataFrame name : str name of SQL table keys : string or sequence, default: None columns to use a primary key con: ADBC Connection, SQLAlchemy connectable, sqlite3 connection, default: None ADBC provides high performance I/O with native type support, where available. Using SQLAlchemy makes it possible to use any DB supported by that library If a DBAPI2 object, only sqlite3 is supported. dtype : dict of column name to SQL type, default None Optional specifying the datatype for columns. The SQL type should be a SQLAlchemy type, or a string for sqlite3 fallback connection. schema: str, default: None Optional specifying the schema to be used in creating the table. r+)rrTrN)rr)rrrrrTrrs r9 get_schemarD s: s # # # z,, 4d%-                    s 8<<)FN)r<r=r>r?)Trh)rir=rjrkrlr!r#)rir=rTrrjrk)rr!rTrrjrkrlr!r).......)rrrrr8rrqrrrrjrrlr!)rrrrr8rrqrrrrjrrlr)rrrrrrrir=r8rrqrrrrjrrlr)rrrrr8rrrrTrrjrrlr!)rrrrr8rrrrTrrjrrlr)rrrir=rrr8rrrrTrrjrrlr).......N) rrrqrrrrjrrTrrlr!) rrrqrrrrjrrTrrlr)rrrir=rqrrrrjrrTrrlr)NrTNNNNr)rrrrrrrr=rrrrrTrrrrrrlr)rrrrrlr=rk)rrrr=rlr)rrrlr)rr/)NNNN)rrrTrrrrlr)pr __future__rabcrr contextlibrrr rr functoolsr rtypingr r rrrrrrhrVpandas._configr pandas._libsrpandas.compat._optionalr pandas.errorsrrpandas.util._exceptionsrpandas.util._validatorsrpandas.core.dtypes.commonrrrrpandas.core.dtypes.dtypesrpandas.core.dtypes.missingrrr pandas.core.apir!r"pandas.core.arraysr#pandas.core.arrays.string_r$pandas.core.baser%pandas.core.commoncorecommonrgr&"pandas.core.internals.constructionr'pandas.core.tools.datetimesr(pandas.io._utilr)collections.abcr*r+rr,sqlalchemy.sql.expressionr-r.pandas._typingr/r0r1r2r3r4r:r[rgrrrrrrrrrr table_existsrrrrrrrrrrlrqrsrrrr;r9rs+  #"""""  ------>>>>>>544444777777 655555++++++322222222222)))))) ......CCCCCC333333111111 !       CGMMMMMD05< *****J!5<4!5< (////F  (+58 #25        (+58 #25      "(,59 $ 25.c?c?c?c?c?L ),3658 25       ),3658 25      ")-3759 !25.e e e e e P ), 25!       ), 25!      $)-  $ 25.!mmmmmh6<%) !15l l l l l ^000002 """"""JPPPPP|PPPfH H H H H cH H H V((((((((&zBCCCC@a'a'a'a'a')a'a'a'NE>E>E>E>E>9E>E>E>V    0000$a$a$a$a$a$(a$a$a$Hy'y'y'y'y'Yy'y'y'~  !        r;