o 2hv@sUdZddlmZddlZddlZddlZddlZddlmZm Z m Z m Z ddl m Z mZddlmZddlmZmZmZmZmZmZmZmZddlmZddlmZmZmZm Z m!Z!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(dd l)m*Z*m+Z+m,Z,erddl-Z-dd l.m/Z/m0Z0m1Z1dd l#m2Z2m3Z3m4Z4m5Z5m6Z6ed e7e4dZ8gdZ9ej:dkrddiniZ;eje>Z?de@d<eZAde@d<[>erddlBmCZCeCdZDedZEeCdZFed ZGGd!d"d"eeDeEeFeGfZHdd'd(ZIed)d*dd/d0ZJe dd)d*dd3d0ZJ dd)d*dd6d0ZJeIeJed7dd?d@ZK[J[Ie ddddAddEdFZLeddGdFZL ddddAddJdFZLddKdLZMejNddPdQZOddUdVZPddYdZZQdd]d^ZRddbdcZSddedfZTddjdkZUddldmZVddqdrZWddwdxZXddydzZYdd{d|ZZdd~dZ[dddZ\dddZ]dddZ^dddZ_dddZ`dddZadddZbdddZcdddZddddZedddZfe7de=e7deQeRe e$jgdehe=eheSeTe!e$jideje=ejeUeVe!e$jkdele=eleWeXee$jmde e=e ecedee$jndee=ee[e\ee$jode e=e e]e^ee$jpde e=e eaebe!e$jqde%e=e%eeefe"e$jrdi Zsde@d<e=eleYeZee$jmdZte=e e_e`ee$jpdZudS)z&Registry for custom pytree node types.) annotationsN) OrderedDict defaultdictdeque namedtuple) itemgetter methodcaller)Lock) TYPE_CHECKINGAnyCallableClassVarGeneric NamedTupleTypeVaroverload) AutoEntry MappingEntryNamedTupleEntry PyTreeEntry SequenceEntryStructSequenceEntry) PyTreeKindStructSequenceTis_namedtuple_classis_structseq_class)safe_ziptotal_order_sortedunzip2) Collection GeneratorIterable)KTVTCustomTreeNode FlattenFunc UnflattenFuncCustomTreeNodeType)bound)register_pytree_noderegister_pytree_node_classunregister_pytree_nodedict_insertion_ordered slotsT)initrepreqfrozenc@sbeZdZUdZded<ded<ded<ejdkrd ed <eZd ed <e j Z d ed<dZ ded<dS)PyTreeNodeRegistryEntryz>A dataclass that stores the information of a pytree node type.zbuiltins.type[Collection[T]]typeFlattenFunc[T] flatten_funcUnflattenFunc[T]unflatten_funcr.zdataclasses.KW_ONLY_zbuiltins.type[PyTreeEntry]path_entry_typerkindstr namespaceN) __name__ __module__ __qualname____doc____annotations__sys version_inforr=rCUSTOMr>rArJrJJ/var/www/html/chatgem/venv/lib/python3.10/site-packages/optree/registry.pyr6@s   r6c@s$eZdZUdZded<d ddZdS) GlobalNamespacerJzClassVar[tuple[()]] __slots__returnr@cCdS)NzrJ)selfrJrJrK__repr__WzGlobalNamespace.__repr__N)rNr@)rBrCrDrMrFrQrJrJrJrKrLTs rLr@__GLOBAL_NAMESPACEr __REGISTRY_LOCK) ParamSpec_P_T_GetP_GetTc@s eZdZdddZdd d ZdS)_CallableWithGetargs_P.argskwargs _P.kwargsrNrWcOtNNotImplementedErrorrPr[r]rJrJrK__call__irRz_CallableWithGet.__call__ _GetP.args _GetP.kwargsrYcOr_r`rarcrJrJrKgetmrRz_CallableWithGet.getN)r[r\r]r^rNrW)r[rer]rfrNrY)rBrCrDrdrgrJrJrJrKrZhs rZrgCallable[_GetP, _GetT]rNDCallable[[Callable[_P, _T]], _CallableWithGet[_P, _T, _GetP, _GetT]]csdfdd }|S)NfuncCallable[_P, _T]rN&_CallableWithGet[_P, _T, _GetP, _GetT]cs |_|Sr`rg)rjrmrJrK decoratorxsz_add_get..decorator)rjrkrNrlrJ)rgrnrJrmrK_add_getqsror?)rAclsr7rAPyTreeNodeRegistryEntry | NonecCdSr`rJrprArJrJrKpytree_node_registry_getrtNone#dict[type, PyTreeNodeRegistryEntry]cCrrr`rJrsrJrJrKrtru type | NoneDdict[type, PyTreeNodeRegistryEntry] | PyTreeNodeRegistryEntry | Nonecs>|turd}|dur|turt|std|dt|ts(td|d|dur^t|dhtfddt D}Wdn1sJwYt |r\t |t<t|t<|S|dkrot ||f}|duro|St |r|turzt S|turtSt |}|dur|St|rt tSt|rt tSdS)aLookup the pytree node registry. >>> register_pytree_node.get() # doctest: +IGNORE_WHITESPACE,ELLIPSIS { : PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ), : PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ), : PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ), ... } >>> register_pytree_node.get(defaultdict) # doctest: +IGNORE_WHITESPACE,ELLIPSIS PyTreeNodeRegistryEntry( type=, flatten_func=, unflatten_func=, path_entry_type=, kind=, namespace='' ) >>> register_pytree_node.get(frozenset) # frozenset is considered as a leaf node None Args: cls (type or None, optional): The class of the pytree node to retrieve. If not provided, all the registered pytree nodes in the namespace are returned. namespace (str, optional): The namespace of the registry to retrieve. If not provided, the global namespace is used. Returns: If the ``cls`` is not provided, a dictionary of all the registered pytree nodes in the namespace is returned. If the ``cls`` is provided, the corresponding registry entry is returned if the ``cls`` is registered as a pytree node. Otherwise, :data:`None` is returned, i.e., the ``cls`` is represented as a leaf node. r?NzExpected a class or None, got .$The namespace must be a string, got csi|] }|jvr|j|qSrJ)rAr7).0handler namespacesrJrK s  z,pytree_node_registry_get..)rSrinspectisclass TypeError isinstancer@ frozensetrT_NODETYPE_REGISTRYvalues_Cis_dict_insertion_ordered&_DICT_INSERTION_ORDERED_REGISTRY_ENTRYdict-_DEFAULTDICT_INSERTION_ORDERED_REGISTRY_ENTRYrrgrrr)rprAregistryr}rJr~rKrtsN<         )r=r9r8r;r:type[Collection[T]]r=type[PyTreeEntry]cCst|s td|dt|rt|tstd|d|tur0t|ts0td|d|dkr8td|turA|}d}n||f}t t |||||t |||||dt |<Wd|S1sgwY|S) aExtend the set of types that are considered internal nodes in pytrees. See also :func:`register_pytree_node_class` and :func:`unregister_pytree_node`. The ``namespace`` argument is used to avoid collisions that occur when different libraries register the same Python type with different behaviors. It is recommended to add a unique prefix to the namespace to avoid conflicts with other libraries. Namespaces can also be used to specify the same class in different namespaces for different use cases. .. warning:: For safety reasons, a ``namespace`` must be specified while registering a custom type. It is used to isolate the behavior of flattening and unflattening a pytree node type. This is to prevent accidental collisions between different libraries that may register the same type. Args: cls (type): A Python type to treat as an internal pytree node. flatten_func (callable): A function to be used during flattening, taking an instance of ``cls`` and returning a triple or optionally a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable metadata to be stored in the treespec and to be passed to the ``unflatten_func``, and (3) (optional) an iterable for the tree path entries to the corresponding children. If the entries are not provided or given by :data:`None`, then `range(len(children))` will be used. unflatten_func (callable): A function taking two arguments: the metadata that was returned by ``flatten_func`` and stored in the treespec, and the unflattened children. The function should return an instance of ``cls``. path_entry_type (type, optional): The type of the path entry to be used in the treespec. (default: :class:`AutoEntry`) namespace (str): A non-empty string that uniquely identifies the namespace of the type registry. This is used to isolate the registry from other modules that might register a different custom behavior for the same type. Returns: The same type as the input ``cls``. Raises: TypeError: If the input type is not a class. TypeError: If the path entry class is not a subclass of :class:`PyTreeEntry`. TypeError: If the namespace is not a string. ValueError: If the namespace is an empty string. ValueError: If the type is already registered in the registry. Examples: >>> # Registry a Python type with lambda functions >>> register_pytree_node( ... set, ... lambda s: (sorted(s), None, None), ... lambda _, children: set(children), ... namespace='set', ... ) >>> # Register a Python type into a namespace >>> import torch >>> register_pytree_node( ... torch.Tensor, ... flatten_func=lambda tensor: ( ... (tensor.cpu().detach().numpy(),), ... {'dtype': tensor.dtype, 'device': tensor.device, 'requires_grad': tensor.requires_grad}, ... ), ... unflatten_func=lambda metadata, children: torch.tensor(children[0], **metadata), ... namespace='torch2numpy', ... ) >>> # doctest: +SKIP >>> tree = {'weight': torch.ones(size=(1, 2)).cuda(), 'bias': torch.zeros(size=(2,))} >>> tree {'weight': tensor([[1., 1.]], device='cuda:0'), 'bias': tensor([0., 0.])} >>> # Flatten without specifying the namespace >>> tree_flatten(tree) # `torch.Tensor`s are leaf nodes ([tensor([0., 0.]), tensor([[1., 1.]], device='cuda:0')], PyTreeSpec({'bias': *, 'weight': *})) >>> # Flatten with the namespace >>> tree_flatten(tree, namespace='torch2numpy') ( [array([0., 0.], dtype=float32), array([[1., 1.]], dtype=float32)], PyTreeSpec( { 'bias': CustomTreeNode(Tensor[{'dtype': torch.float32, 'device': device(type='cpu'), 'requires_grad': False}], [*]), 'weight': CustomTreeNode(Tensor[{'dtype': torch.float32, 'device': device(type='cuda', index=0), 'requires_grad': False}], [*]) }, namespace='torch2numpy' ) ) >>> # Register the same type with a different namespace for different behaviors >>> def tensor2flatparam(tensor): ... return [torch.nn.Parameter(tensor.reshape(-1))], tensor.shape, None ... ... def flatparam2tensor(metadata, children): ... return children[0].reshape(metadata) ... ... register_pytree_node( ... torch.Tensor, ... flatten_func=tensor2flatparam, ... unflatten_func=flatparam2tensor, ... namespace='tensor2flatparam', ... ) >>> # Flatten with the new namespace >>> tree_flatten(tree, namespace='tensor2flatparam') ( [ Parameter containing: tensor([0., 0.], requires_grad=True), Parameter containing: tensor([1., 1.], device='cuda:0', requires_grad=True) ], PyTreeSpec( { 'bias': CustomTreeNode(Tensor[torch.Size([2])], [*]), 'weight': CustomTreeNode(Tensor[torch.Size([1, 2])], [*]) }, namespace='tensor2flatparam' ) ) Expected a class, got rz(Expected a subclass of PyTreeEntry, got r{r?(The namespace cannot be an empty string.r=rAN)rrr issubclassrrSrr@ ValueErrorrTr register_noder6r)rpr9r;r=rAregistration_keyrJrJrKr*s>  r*r str | Nonetype[PyTreeEntry] | None2Callable[[CustomTreeNodeType], CustomTreeNodeType]cCrrr`rJrpr=rArJrJrKr+r+cCrrr`rJrrJrJrKr+rCustomTreeNodeType | str | NoneGCustomTreeNodeType | Callable[[CustomTreeNodeType], CustomTreeNodeType]cs|tus t|trdurtd|dkrtdd|}dur&tdtur6tts6tddkr>td|durLdfd d }|St|sYtd |d durct|dttrmt t sutdd t |t d|j d|S)a Extend the set of types that are considered internal nodes in pytrees. See also :func:`register_pytree_node` and :func:`unregister_pytree_node`. The ``namespace`` argument is used to avoid collisions that occur when different libraries register the same Python type with different behaviors. It is recommended to add a unique prefix to the namespace to avoid conflicts with other libraries. Namespaces can also be used to specify the same class in different namespaces for different use cases. .. warning:: For safety reasons, a ``namespace`` must be specified while registering a custom type. It is used to isolate the behavior of flattening and unflattening a pytree node type. This is to prevent accidental collisions between different libraries that may register the same type. Args: cls (type, optional): A Python type to treat as an internal pytree node. path_entry_type (type, optional): The type of the path entry to be used in the treespec. (default: :class:`AutoEntry`) namespace (str, optional): A non-empty string that uniquely identifies the namespace of the type registry. This is used to isolate the registry from other modules that might register a different custom behavior for the same type. Returns: The same type as the input ``cls`` if the argument presents. Otherwise, return a decorator function that registers the class as a pytree node. Raises: TypeError: If the path entry class is not a subclass of :class:`PyTreeEntry`. TypeError: If the namespace is not a string. ValueError: If the namespace is an empty string. ValueError: If the type is already registered in the registry. This function is a thin wrapper around :func:`register_pytree_node`, and provides a class-oriented interface: .. code-block:: python @register_pytree_node_class(namespace='foo') class Special: TREE_PATH_ENTRY_TYPE = GetAttrEntry def __init__(self, x, y): self.x = x self.y = y def tree_flatten(self): return ((self.x, self.y), None, ('x', 'y')) @classmethod def tree_unflatten(cls, metadata, children): return cls(*children) @register_pytree_node_class('mylist') class MyList(UserList): TREE_PATH_ENTRY_TYPE = SequenceEntry def tree_flatten(self): return self.data, None, None @classmethod def tree_unflatten(cls, metadata, children): return cls(*children) Nz?Cannot specify `namespace` when the first argument is a string.r?rz.decoratorrrzTREE_PATH_ENTRY_TYPEr tree_flattenr)rpr(rNr()rSrr@rrrrgetattrrrrr*rtree_unflatten)rpr=rArnrJrrKr+s:F   cCst|s td|d|turt|tstd|d|dkr&td|tur/|}d}n||f}tt ||t |WdS1sKwYdS)aRemove a type from the pytree node registry. See also :func:`register_pytree_node` and :func:`register_pytree_node_class`. This function is the inverse operation of function :func:`register_pytree_node`. Args: cls (type): A Python type to remove from the pytree node registry. namespace (str): The namespace of the pytree node registry to remove the type from. Returns: The removed registry entry. Raises: TypeError: If the input type is not a class. TypeError: If the namespace is not a string. ValueError: If the namespace is an empty string. ValueError: If the type is a built-in type that cannot be unregistered. ValueError: If the type is not found in the registry. Examples: >>> # Register a Python type with lambda functions >>> register_pytree_node( ... set, ... lambda s: (sorted(s), None, None), ... lambda _, children: set(children), ... namespace='temp', ... ) >>> # Unregister the Python type >>> unregister_pytree_node(set, namespace='temp') rrzr{r?rN) rrrrSrr@rrTrunregister_noderpop)rprArrJrJrKr,%s " $r,modeboolGenerator[None]ccs|turt|tstd|d|dkrtd|tur d}ttj|dd}tt ||Wdn1s>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_flatten(tree) # doctest: +IGNORE_WHITESPACE ( [1, 2, 3, 4, 5], PyTreeSpec({'a': *, 'b': (*, [*, *]), 'c': None, 'd': *}) ) >>> with dict_insertion_ordered(True, namespace='some-namespace'): # doctest: +IGNORE_WHITESPACE ... tree_flatten(tree, namespace='some-namespace') ( [2, 3, 4, 1, 5], PyTreeSpec({'b': (*, [*, *]), 'a': *, 'c': None, 'd': *}, namespace='some-namespace') ) .. warning:: The dictionary sorting mode is a global setting and is **not thread-safe**. It is recommended to use this context manager in a single-threaded environment. Args: mode (bool): The dictionary sorting mode to set. namespace (str): The namespace to set the dictionary sorting mode for. r{rzr?rF)inherit_global_namespaceN) rSrr@rrrTrrset_dict_insertion_orderedr)rrAprevrJrJrKr-Zs$(r-itemsIterable[tuple[KT, VT]]list[tuple[KT, VT]]cCst|tddS)Nr)key)rr)rrJrJrK _sorted_itemssrr<tuple[tuple[()], None]cCrO)N)rJNrJ)r<rJrJrK _none_flattenrRrchildren Iterable[Any]cCs$t}tt|||urtddS)NzExpected no children.)objectnextiterr)r<rsentinelrJrJrK_none_unflattensrtup tuple[T, ...]tuple[tuple[T, ...], None]cC|dfSr`rJrrJrJrK_tuple_flattenr Iterable[T]cCt|Sr`)tupler<rrJrJrK_tuple_unflattenrrlstlist[T]tuple[list[T], None]cCrr`rJ)rrJrJrK _list_flattenrrcCrr`)listrrJrJrK_list_unflattenrrdct dict[KT, VT]/tuple[tuple[VT, ...], list[KT], tuple[KT, ...]]cCs"tt|\}}|t||fSr`)rrrrrkeysrrJrJrK _dict_flattensrrlist[KT]r Iterable[VT]cCtt||Sr`rrrrrJrJrK_dict_unflattenrcCt|\}}|t||fSr`rrrrrJrJrK_dict_insertion_ordered_flattenrcCrr`rrrJrJrK!_dict_insertion_ordered_unflattenrrOrderedDict[KT, VT]cCrr`rrrJrJrK_ordereddict_flattenrrcCrr`)rrrrJrJrK_ordereddict_unflattenrrdefaultdict[KT, VT]Otuple[tuple[VT, ...], tuple[Callable[[], VT] | None, list[KT]], tuple[KT, ...]]cCt|\}}}||j|f|fSr`)rdefault_factoryrrrentriesrJrJrK_defaultdict_flattenrmetadata!tuple[Callable[[], VT], list[KT]]cC|\}}t|t||Sr`)rrrrrrrJrJrK_defaultdict_unflattenrcCrr`)rrrrJrJrK&_defaultdict_insertion_ordered_flattenrrcCrr`)rrrrJrJrK(_defaultdict_insertion_ordered_unflattenrrdeqdeque[T]tuple[deque[T], int | None]cCs ||jfSr`maxlen)rrJrJrK_deque_flattens rr int | NonecCs t||dS)Nr)r)rrrJrJrK_deque_unflatten r NamedTuple[T])tuple[tuple[T, ...], type[NamedTuple[T]]]cC |t|fSr`r7rrJrJrK_namedtuple_flattenrrtype[NamedTuple[T]]cCs||Sr`rJrprrJrJrK_namedtuple_unflattenrrseqStructSequence[T]-tuple[tuple[T, ...], type[StructSequence[T]]]cCrr`r)rrJrJrK_structseq_flatten rrtype[StructSequence[T]]cCs||Sr`rJrrJrJrK_structseq_unflattensr)r=r>z6dict[type | tuple[str, type], PyTreeNodeRegistryEntry]rrJ)rgrhrNri)rpr7rAr@rNrqr`)rprvrAr@rNrw)rprxrAr@rNry) r9r8r;r:rprr=rrAr@rNr)rprr=rrArrNr)rpr(r=rrAr@rNr()rprr=rrArrNr)rpr7rAr@rNr6)rrrAr@rNr)rrrNr)r<rvrNr)r<rvrrrNrv)rrrNr)r<rvrrrNr)rrrNr)r<rvrrrNr)rrrNr)rrrrrNr)rrrNr)rrrrrNr)rrrNr)rrrrrNr)rrrNr)rrrrrNr)rrrNr)rprrrrNr)rrrNr)rprrrrNr)vrE __future__r contextlib dataclassesrrG collectionsrrrroperatorrr threadingr typingr r r r rrrr optree._Croptree.accessorsrrrrrr optree.typingrrrrr optree.utilsrrrbuiltinscollections.abcr r!r"r#r$r%r&r'r7r(__all__rHSLOTS dataclassr6rLrSrFrTtyping_extensionsrUrVrWrXrYrZrortr*r+r,contextmanagerr-rrrrrrrrrrrrrrrrrrrrrrrNONErTUPLErLISTrDICT NAMEDTUPLE ORDEREDDICT DEFAULTDICTDEQUESTRUCTSEQUENCErrrrJrJrJrKsT  (     k!   p5  .                  C