관리-도구
편집 파일: horizontal_shard.cpython-38.pyc
U -?�fnA � @ sv d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddlm Z dd lm Z dd lm Z dd lm Z dd lm Z d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlm Z d dl m Z d dl m Z d dl m Z d dl!m"Z" d dl!m#Z# e �r�d d l$m%Z% d d l$m&Z& d d l$m'Z' d d l(m)Z) d d l(m*Z* d d!lm+Z+ d d"lm,Z, d d#l-m.Z. d d$l/m0Z0 d d%l m1Z1 d d&l m2Z2 d d'l m3Z3 d d(l4m5Z5 d d)l6m7Z7 d d*l8m9Z9 d d+l:m;Z; d,d-gZ<e d.ed/�Z=e>Z?G d0d1� d1e"�Z@G d2d3� d3e"�ZAG d4d-� d-e e= �ZBG d5d,� d,e �ZCG d6d7� d7e�ZDd8d9d:�d;d<�ZEd=S )>a� Horizontal sharding support. Defines a rudimental 'horizontal sharding' system which allows a Session to distribute queries and persistence operations across multiple databases. For a usage example, see the :ref:`examples_sharding` example included in the source distribution. .. deepalchemy:: The horizontal sharding extension is an advanced feature, involving a complex statement -> database interaction as well as use of semi-public APIs for non-trivial cases. Simpler approaches to refering to multiple database "shards", most commonly using a distinct :class:`_orm.Session` per "shard", should always be considered first before using this more complex and less-production-tested system. � )� annotations)�Any)�Callable)�Dict)�Iterable)�Optional)�Tuple)�Type)� TYPE_CHECKING)�TypeVar)�Union� )�event)�exc)�inspect)�util)� PassiveFlag)�OrmExecuteOptionsParameter)� ORMOption)�Mapper)�Query)�_BindArguments)�_PKIdentityArgument)�Session)�Protocol)�Self)� Connection)�Engine)� OptionEngine)�IteratorResult)�Result)�LoaderCallableStatus)�_O)�BulkUDCompileState)� QueryContext)�_EntityBindKey)� _SessionBind)�ORMExecuteState)� InstanceState)� Executable)�_TP)� ClauseElement�ShardedSession� ShardedQuery�_T)�boundc @ s e Zd Zddddd�dd�ZdS )� ShardChooser�Optional[Mapper[_T]]r �Optional[ClauseElement])�mapper�instance�clause�returnc C s d S �N� )�selfr3 r4 r5 r8 r8 �O/opt/hc_python/lib64/python3.8/site-packages/sqlalchemy/ext/horizontal_shard.py�__call__Q s zShardChooser.__call__N��__name__� __module__� __qualname__r; r8 r8 r8 r: r0 P s r0 c @ s& e Zd Zdddddddd�dd �Zd S ) �IdentityChooser� Mapper[_T]r � Optional[InstanceState[Any]]r r r �r3 � primary_key�lazy_loaded_from�execution_options�bind_arguments�kwr6 c K s d S r7 r8 )r9 r3 rD rE rF rG rH r8 r8 r: r; Z s zIdentityChooser.__call__Nr< r8 r8 r8 r: r@ Y s r@ c s: e Zd ZdZdddd�� fdd� Zddd �d d �Z� ZS ) r- a Query class used with :class:`.ShardedSession`. .. legacy:: The :class:`.ShardedQuery` is a subclass of the legacy :class:`.Query` class. The :class:`.ShardedSession` now supports 2.0 style execution via the :meth:`.ShardedSession.execute` method. r �None)�args�kwargsr6 c s< t � j||� t| jt�s t�| jj| _| jj| _d | _d S r7 ) �super�__init__� isinstance�sessionr, �AssertionError�identity_chooser�execute_chooserZ _shard_id)r9 rJ rK �� __class__r8 r: rM o s zShardedQuery.__init__�ShardIdentifierr ��shard_idr6 c C s | j |d�S )a� Return a new query, limited to a single shard ID. All subsequent operations with the returned query will be against the single shard regardless of other state. The shard_id can be passed for a 2.0 style execution to the bind_arguments dictionary of :meth:`.Session.execute`:: results = session.execute( stmt, bind_arguments={"shard_id": "my_shard"} ) )� _sa_shard_id)rF )r9 rW r8 r8 r: � set_shardw s zShardedQuery.set_shard)r= r> r? �__doc__rM rY � __classcell__r8 r8 rS r: r- f s c s� e Zd ZU ded< ded< ded< dddefddd�dd d d d d dddd� � fdd�Zdejdej dfdddddddddd � � fd d � Z d dddd �d!d"�Z d4d#dd$dd%d&�d'd(�Z d5dddd)�d d$dd*dd+d,�d-d.�Z d/d0dd1�d2d3�Z� ZS )6r, r0 � shard_chooserr@ rQ z*Callable[[ORMExecuteState], Iterable[Any]]rR N)� id_chooser� query_chooserzOptional[IdentityChooser]z4Optional[Callable[[ORMExecuteState], Iterable[Any]]]zOptional[Dict[str, Any]]zType[Query[_T]]z<Optional[Callable[[Query[_T], Iterable[_T]], Iterable[Any]]]z/Optional[Callable[[Executable], Iterable[Any]]]r rI ) r\ rQ rR �shards� query_clsr] r^ rK r6 c s t � jf d|i|�� tj�dtdd� |�_|rj|� t�dd� ddd d d d d d �� �fdd� } | �_n|rv|�_n t � d��|r�|�t�dd� |r�t � d��ddd��fdd� } |dkr�| }|dkr�t � d��|�_ i �_ |dk �r|D ]} �� | || � q�dS )a� Construct a ShardedSession. :param shard_chooser: A callable which, passed a Mapper, a mapped instance, and possibly a SQL clause, returns a shard ID. This id may be based off of the attributes present within the object, or on some round-robin scheme. If the scheme is based on a selection, it should set whatever state on the instance to mark it in the future as participating in that shard. :param identity_chooser: A callable, passed a Mapper and primary key argument, which should return a list of shard ids where this primary key might reside. .. versionchanged:: 2.0 The ``identity_chooser`` parameter supersedes the ``id_chooser`` parameter. :param execute_chooser: For a given :class:`.ORMExecuteState`, returns the list of shard_ids where the query should be issued. Results from all shards returned will be combined together into a single listing. .. versionchanged:: 1.4 The ``execute_chooser`` parameter supersedes the ``query_chooser`` parameter. :param shards: A dictionary of string shard names to :class:`~sqlalchemy.engine.Engine` objects. r` Zdo_orm_executeT)�retvalzLThe ``id_chooser`` parameter is deprecated; please use ``identity_chooser``.z2.0rA r rB r r r rC c s" �� | �}|r|�|�}� ||�S r7 )�queryZ_set_lazyload_from)r3 rD rE rF rG rH �q)� _id_chooserr9 r8 r: �_legacy_identity_chooser� s z9ShardedSession.__init__.<locals>._legacy_identity_chooserz*identity_chooser or id_chooser is requiredzNThe ``query_chooser`` parameter is deprecated; please use ``execute_chooser``.z1.4z>Can't pass query_chooser and execute_chooser at the same time.r' z Iterable[Any]�� orm_contextr6 c s � | j �S r7 )Z statement�rg )�_query_chooserr8 r: �_default_execute_chooser� s z9ShardedSession.__init__.<locals>._default_execute_chooserNz,execute_chooser or query_chooser is required)rL rM r �listen�execute_and_instancesr\ r Zwarn_deprecatedrQ r � ArgumentErrorrR �_ShardedSession__shards� bind_shard) r9 r\ rQ rR r_ r` r] r^ rK re rj �krS )rd ri r9 r: rM � sV , �� ���� zShardedSession.__init__z Mapper[_O]zUnion[Any, Tuple[Any, ...]]z Optional[Any]r rB r zOptional[_BindArguments]z)Union[Optional[_O], LoaderCallableStatus]) r3 �primary_key_identity�identity_token�passiverE rF rG rH r6 c s� |dk r&t � j||fd|i|��} | S | j|||||r>t|�ni d�D ]0} t � j||f| |d�|��} | dk rF| S qFdS dS )a_ override the default :meth:`.Session._identity_lookup` method so that we search for a given non-token primary key identity across all possible identity tokens (e.g. shard ids). .. versionchanged:: 1.4 Moved :meth:`.Session._identity_lookup` from the :class:`_query.Query` object to the :class:`.Session`. Nrr )rE rF rG )rr rE )rL �_identity_lookuprQ �dict) r9 r3 rq rr rs rE rF rG rH �objrW �obj2rS r8 r: rt � s: ���� ��� z ShardedSession._identity_lookupz Optional[_EntityBindKey[_O]])r3 r4 rH r6 c K sl |d k r<t |�}|jr0|jd }|d k s,t�|S |jr<|jS t|t�sJt�| j||f|�}|d k rh||_|S )Nr )r �keyrP rr rN r r\ )r9 r3 r4 rH �state�tokenrW r8 r8 r: �_choose_shard_and_assign/ s z'ShardedSession._choose_shard_and_assignr1 zOptional[ShardIdentifier]r )r3 r4 rW rH r6 c K sz |dkr| � ||�}| �� r>| �� }|dk s0t�|j||d�S | j|||d�}t|t�rd|jf |�S t|t �srt�|S dS )zaProvide a :class:`_engine.Connection` to use in the unit of work flush process. N)rW )r3 rW r4 ) r{ Zin_transactionZget_transactionrP � connection�get_bindrN r �connectr )r9 r3 r4 rW rH Ztrans�bindr8 r8 r: �connection_callableD s � z"ShardedSession.connection_callable)rW r4 r5 r2 r& )r3 rW r4 r5 rH r6 c K s. |d kr$| j |||d�}|d k s$t�| j| S )N)r4 r5 )r{ rP rn )r9 r3 rW r4 r5 rH r8 r8 r: r} b s � zShardedSession.get_bindrU zUnion[Engine, OptionEngine])rW r r6 c C s || j |< d S r7 )rn )r9 rW r r8 r8 r: ro r s zShardedSession.bind_shard)NNN)N)r= r> r? �__annotations__r- rM r Z PASSIVE_OFFr � EMPTY_DICTrt r{ r� r} ro r[ r8 r8 rS r: r, � s: � �&t�$1 � �� c @ s& e Zd ZdZdZd ddd�dd�ZdS ) � set_shard_ida a loader option for statements to apply a specific shard id to the primary query as well as for additional relationship and column loaders. The :class:`_horizontal.set_shard_id` option may be applied using the :meth:`_sql.Executable.options` method of any executable statement:: stmt = ( select(MyObject). where(MyObject.name == 'some name'). options(set_shard_id("shard1")) ) Above, the statement when invoked will limit to the "shard1" shard identifier for the primary query as well as for all relationship and column loading strategies, including eager loaders such as :func:`_orm.selectinload`, deferred column loaders like :func:`_orm.defer`, and the lazy relationship loader :func:`_orm.lazyload`. In this way, the :class:`_horizontal.set_shard_id` option has much wider scope than using the "shard_id" argument within the :paramref:`_orm.Session.execute.bind_arguments` dictionary. .. versionadded:: 2.0.0 �rW �propagate_to_loadersTrU �boolc C s || _ || _dS )aH Construct a :class:`_horizontal.set_shard_id` option. :param shard_id: shard identifier :param propagate_to_loaders: if left at its default of ``True``, the shard option will take place for lazy loaders such as :func:`_orm.lazyload` and :func:`_orm.defer`; if False, the option will not be propagated to loaded objects. Note that :func:`_orm.defer` always limits to the shard_id of the parent row in any case, so the parameter only has a net effect on the behavior of the :func:`_orm.lazyload` strategy. Nr� )r9 rW r� r8 r8 r: rM � s zset_shard_id.__init__N)T)r= r> r? rZ � __slots__rM r8 r8 r8 r: r� x s �r� r' �&Union[Result[_T], IteratorResult[_TP]]rf c s � j r� j}n� js� jr"� j}nd }� j}t|t�s:t�ddd�� fdd� }� j D ]}t|t �rT|j } q�qT|r�|j d k r�|j }n0d� j kr�� j d }nd� jkr�� jd }nd }|d k r�||�S g }|�� �D ]}||�}|�|� q�|d j|d d � � S d S ) NrU r� rV c s* t � j�}| |d< � j| d� � j|d�S )NrW )rr )rG )ru rG Zupdate_execution_optionsZinvoke_statement)rW rG rh r8 r: �iter_for_shard� s z-execute_and_instances.<locals>.iter_for_shardrX rW r � )Z is_selectZ load_optionsZ is_updateZ is_deleteZupdate_delete_optionsrO rN r, rP Z_non_compile_orm_optionsr� rW Z_identity_tokenrF rG rR �append�merge)rg Zactive_optionsrO r� Zorm_optrW �partialZresult_r8 rh r: rl � s4 rl N)FrZ � __future__r �typingr r r r r r r r r r � r r r r Zormr Z orm._typingr Zorm.interfacesr Z orm.mapperr Z orm.queryr Z orm.sessionr r r Z util.typingr r Z engine.baser r r Z engine.resultr r r! r" Zorm.bulk_persistencer# Z orm.contextr$ r% r& r' Z orm.stater( �sqlr) Z sql._typingr* Z sql.elementsr+ �__all__r. �strrU r0 r@ r- r, r� rl r8 r8 r8 r: �<module> sh # p2