o 2h@sXdZddlZddlmZmZmZddlmZddlmZddlm Z ddlm Z ddlm Z dd l m Z dd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddl!m"Z"Gddde#Z$ddZ%ddZ&ddZ'e(dgd Z)d!d"Z*d#d$Z+d%d&Z,dS)'z7Logic for restoring checkpointed values for Trackables.N)OptionalMappingAny)checkpoint_adapter)checkpoint_view)functional_saver) save_util_v1)saveable_compat)context)ops)tensor) array_ops) gen_io_ops)io_ops) tf_logging) registration)base) constants) python_state)trackable_utils)saveable_object_util)object_identityc @sDeZdZdZgdZddZdIndicates a position within a `_CheckpointRestoreCoordinator`.) _checkpoint _proto_id skip_restorecallbackcCs ||_||_d|_t|_dS)zSpecify an object within a checkpoint. Args: checkpoint: A _CheckpointRestoreCoordinator object. proto_id: The index of this object in TrackableObjectGraph.nodes. FN)rrrrReshardCallbackr)self checkpointproto_idr!_/var/www/html/chatgem/venv/lib/python3.10/site-packages/tensorflow/python/checkpoint/restore.py__init__.szCheckpointPosition.__init__NcCsrt+||r||}|r'|j|WddSWddSWddS1s2wYdS)z$Restore this value into `trackable`.N)r init_scope bind_object_restore_descendantsrnew_restore_ops)r trackablereader restore_opsr!r!r"restore<s   "zCheckpointPosition.restorecCsl|j}|j||j|jd}|j|j|dur$||j|j<dS||ur4td|d|ddS)a\Set a checkpoint<->object correspondence. Args: trackable: The object to record a correspondence for. Returns: True if this is a new assignment, False if this object has already been mapped to a checkpointed `Object` proto. Raises: AssertionError: If another object is already bound to the `Object` proto. NTa&Inconsistent references when loading the checkpoint into this object graph. For example, in the saved checkpoint object, `model.layer.weight` and `model.layer_copy.weight` reference the same variable, while in the current object these are two different variables. The referenced variables are:(z and z).F) rall_python_objectsaddobject_by_proto_idgetrmatched_proto_idsloggingwarning)rr(rcurrent_assignmentr!r!r"r%Fs   zCheckpointPosition.bind_objectrcCs$ttjt|js td||_dS)zAdd a resharding callback to the checkpoint. This will be applied to the checkpoint value before being supplied to the restore ops. Args: callback: Reshard callback for resharding this checkpoint position. Maybe None. z@Cannot override resharding callback, already set to non trivial.N) issubclassrrtyper TypeError)rrr!r!r"update_resharding_callbackhs   z-CheckpointPosition.update_resharding_callbackreturncCsttjt|j S)zCDetermine whether this value has a non-trivial resharding callback.)r4rrr5rrr!r!r" has_non_trivial_reshard_callbackzs z3CheckpointPosition.has_non_trivial_reshard_callbackcCs.|jj}t|dko|djtjko|jj S)zEDetermine whether this value is restorable with a Tensor initializer.r) object_proto attributeslennamerVARIABLE_VALUE_KEYchildren)rr=r!r!r"is_simple_variables  z%CheckpointPosition.is_simple_variableshape_and_slicesc Csi}|jjD]}|j}|jjjpd}tit|L|dur,|j |vr,||j }nd}|j ||\}}g} |D]} |jj | } | | jq;tj|jj||| d|j fd} |j | |} Wdn1skwYt| ||j <Wdn1swYq|S)aCreate value `Tensor`s for this object's attributes. Does not require that the Python object has been created. Used for restore-on-create when executing eagerly. Args: shape_and_slices: A dict mapping from object attribute names to a shape and slice string that will be passed to a RestoreV2 op. If the dict is None or if an object attribute is not in the dict, the full tensor will be restored. Returns: A dictionary mapping from object attribute names to `Tensor`s. zcpu:0Nz%s_checkpoint_read)prefix tensor_namesrCdtypesr?)r<r=checkpoint_keyroptionsexperimental_io_devicer r$devicer?rupdate_restore_inputs dtype_mapappend base_dtyper restore_v2save_path_tensorreshardr identity)rrC value_tensorsserialized_tensorrH io_deviceshape_and_slicecheckpoint_keysfull_shape_and_slicesrGkeydtyperestored_valuesvaluer!r!r"rTsB       z CheckpointPosition.value_tensorsc Cs|}|jjs|sgigifSg}i}g}tt}t|j}t |j}|r9|j s8|jj j }|j|||<nJ|rH|jjdj |ji||<n;t|jtjrU||n.|tjhkre||\}}n|ro||\}}n|jjD]} |jj|jg| jqs||||fS)zLooks up or creates SaveableObjects which don't have cached ops. Returns: A tuple of ( existing_restore_ops: list, named_saveables: dict, python_positions: list, registered_savers: dict) r)get_registered_saver_namer<r= collections defaultdictdictrsaveable_objects_from_trackabler(rrregistered_saver object_namerH isinstancer PythonStaterNkeysrSERIALIZE_TO_TENSORS_NAME$_create_serialize_to_tensor_saveable#_create_saveables_by_attribute_namerunused_attributes setdefaultrr?) rrecorded_registered_saverexisting_restore_opsnamed_saveablespython_positionsregistered_saverssaveable_factories saver_namer?rUr!r!r"gather_ops_or_named_saveablessJ         z0CheckpointPosition.gather_ops_or_named_saveablescCst|jpd}t|jjdj|}ts=|j j |d}|dur(|gifS|j j |ji}||vr=g|||ifS|tj|d}tsM|||<g||ifS)z9Creates a saveable using the _serialize_to_tensor method.rDrNr?)r get_saveable_namer(_extract_saveable_namer<r=rHr executing_eagerlyrrestore_ops_by_namer/saveables_cacherlrrh)rrrsuffix saveable_name existing_oprzsaveabler!r!r"ris.   z7CheckpointPosition._create_serialize_to_tensor_saveablec s6i}g}t}|jjD]trd}n |jjjd}|dur'| |q t fdd|Dr3q |jj }|dur>d}n||j ij d}t|dkrT|\}nd}|durfj|j vrfd}||j =|durt||}|dur|jj|jg j q |dur|g||j ij <||j<q ||fS)a+Creates or caches SaveableObjects by matching the attribute names. The attribute name keys in the `saveable_factories` is used to find the corresponding attribute in the object proto. Attributes contain checkpoint keys which are passed to the factory function to generate the SaveableObject. Args: saveable_factories: a dict mapping attribute name to a callable factory function that produces a SaveableObject. Returns: A tuple of ( existing_restore_ops: list, named_saveables: dict) Nc3s|] }j|VqdSN)r? startswith).0r?rUr!r" :szICheckpointPosition._create_saveables_by_attribute_name..rr;)setr<r=r rxrryr/rHrNanyrzr(r?r>_get_saveable_from_factoryrkrlr) rrrrorncreated_compat_namesr}rzr~ saveable_listr!rr"rjsj       z6CheckpointPosition._create_saveables_by_attribute_namecCs<|rtd|\}}}}||jj|||d|S)ajCreate or fetch restore ops for this object's attributes. Requires that the `Trackable` Python object has been bound to an object ID in the checkpoint. Args: reader: A `CheckpointReader`. If None, a new instance will be created. Returns: A list of operations when graph building, or an empty list when executing eagerly. zOUnable to run individual checkpoint restore for objects with registered savers.r))_has_registered_saver ValueErrorrtextendrrestore_saveables)rr)r*tensor_saveablesrp_r!r!r"r*ps zCheckpointPosition.restore_opscC|jSr)rr9r!r!r"rzCheckpointPosition.checkpointcCs|jj|jSr)rr.rr9r!r!r"r(szCheckpointPosition.trackablecCs|jjj|jSr)robject_graph_protonodesrr9r!r!r"r<szCheckpointPosition.object_protocCrr)rr9r!r!r"r rzCheckpointPosition.proto_idcCs|jjSr)r restore_uidr9r!r!r"rszCheckpointPosition.restore_uidcCs t|jSr)reprr<r9r!r!r"__repr__s zCheckpointPosition.__repr__cCs0|jjD]}|jtjkr|jj|jSqdS)zoThe shape of the VARIABLE_VALUE tensor. Returns: If found a TensorShape object, otherwise None. N)r<r=r?rr@r shape_maprH)rrUr!r!r" value_shapes  zCheckpointPosition.value_shapecCst|jjjSr)boolr<rcr?r9r!r!r"rz(CheckpointPosition._has_registered_saverc Csf|r1|jjj}z t|j|W|Sty0}zt|r"|d|_ WYd}~|Sd}~wwdS)z Y       3  ,rc Cs|durtdt|tstdt|}|}|D]0\}}||vs.|jj|dur9td|d|d|dusCt|t j sNtd|d|d qt }|D]\}}|jj|j }|jj|} t|} | s| j st|rtd |d |d qWt } t| j d j| |<t| d\} } t| d }t| j t|krtd t|dt| j d t||}|||<t||}|r|std |d |d |s|rtd |d|d|d t| j t||kr tdt||dt| j d | st||qWt }i||<| j D]}|j}|j}||||||<q"t||qWdS)aRestores nodes from a dict. Requires that the `Trackable` Python object has been bound to an object ID in the checkpoint. Args: save_path: a string represents path to the checkpoint. nodes_to_restore: a dict maps `node_id` to `trackable` to be restored. Nzsave_path cannot be empty.zDExpecting a dictionary of node_id to Trackable for nodes_to_restore.zThe expected node_id: z to Trackable z- to restore does not exist in the checkpoint.z(Expecting a valid Trackable to node_id: z but got trackable: .z Trackable z] expects checkpointed values but checkpoint does not contain serialized tensors for node_id: rz)Size for saveable_objects for Trackable: z? did not match the size for serialized_tensors for checkpoint: zQ does not expect checkpointed values but checkpoint contains serialized tensors: z for node_id: z+Size for serialized_tensors for Trackable: z; did not match size for serialized_tensors for checkpoint: )rrerarCheckpointView descendantsr_object_graph_protorrrrObjectIdentityDictionaryr=r!trackable_has_serialize_to_tensorrbrextract_object_namerHr!get_checkpoint_factories_and_keysgenerate_saveable_objectsr>SaveableCompatibilityConverter_serialize_to_tensorsrrMultiDeviceSaverr+r?) save_pathnodes_to_restore ckpt_viewckpt_view_descendantsrr(serialized_tensorscurrent_trackable ckpt_contains_serialized_tensorsnoder object_namescheckpoint_factory_maprsaveable_objectstrackable_expects_ckpted_valueserialized_tensors_renamed attributer?rHr!r!r" restore_nodes/s             rcCs$||jj}|r||r|SdSr)_checkpoint_adapterrsave_path_string is_applicable)checkpoint_positionr(adapterr!r!r"_maybe_get_adapters rc Cs|j}|}t||}|jjD]l}|jj|jd}|dur q| |j}| |j |}|j} |duro| dr>| j j} nt| jpL| jpL| jpL| d} | rn|j } |rd||j \} } | rd|| |j| g|q|j|dr||||fqdS)z>Queues the restoration of trackable's children or defers them.Nhas_checkpoint_valuesrc)r()r(_trackable_childrenrr<rArr.r/rr_lookup_dependency local_nameHasFieldrr]rr=slot_variables maybe_reshardr7_deferred_dependenciesrlrNr%) rrr(trackable_childrenrchildcorrespondencechild_position local_object child_proto has_valuelocal_trackable_namerr!r!r"rsT       r _DeferredSlotVariableRestorationoriginal_variablerrc Cs|j}|j}|j|jdD]"}|||j|j|j| r |j nd\}}|dur0| ||fq|j |jdD]E}|j |jd}|durZ|j|jg t||j|jdq9t|dr~||||j|j| rn|j nd\}}|dur~| ||fq9dS)z&Queues slot variables for restoration.r!Nrr)r(rdeferred_slot_restorationspopr rrrrr:rrNslot_restorationsr.r/ optimizer_idrlrhasattr) rrr(rdeferred_slot_restorationrrslot_restorationrr!r!r"rsd rcCs$tjd}|d||t|S)N/)rOBJECT_ATTRIBUTES_NAMEindexr>)rH search_keyr!r!r"rws rwcCsd}|j}|j}||vr||}|dur8|D]\}}||r7|dur)td|}t|j|}||qt|rA||dS|S)z7Returns the saveable generated from the factory method.NzForward compatibility load error: Unable to load checkpoint saved in future version of TensorFlow. Please update your version of TensorFlow to the version in which the checkpoint was saved.ru)r?rHrrrrwr-callable)rrrUrmatched_factoryexpected_factory_namefactory_input_name factory_namefactoryr!r!r"r s*   r)-rr_typingrrrtensorflow.python.checkpointrrrrr tensorflow.python.eagerr tensorflow.python.frameworkr r tensorflow.python.opsr rrtensorflow.python.platformrr1tensorflow.python.saved_modelrtensorflow.python.trackablerrrr!tensorflow.python.training.savingrtensorflow.python.utilrobjectrrrr namedtuplerrrwrr!r!r!r"sH                    k 4;