o 2ha@sdZddlZddlZddlmZddlmZddlmZddlm Z ddl m Z ddl m Z dd lmZd Zd Zd d ZddZ  dddZ  dddZedgddddZdS)zqLogging tensorflow::tfprof::OpLogProto. OpLogProto is used to add extra model information for offline analysis. N)tfprof_log_pb2)context)ops) tensor_shape)gfile)flops_registry) tf_export_trainable_variablesflopsc Cs|jjD]}|jD]~}|jsq z||j}Wnty)}zWYd}~q d}~wwt|jt|jkr5q t |jD]L\}}|j| rHq:|j j j}tdd|D} z|j||j| | Wq:ty}ztjd|j|fWYd}~q:d}~wwq q|S)zBFill Tensor shapes in 'graph' with run time shape from 'run_meta'.NcSsg|]}|jqS)size).0dr r c/var/www/html/chatgem/venv/lib/python3.10/site-packages/tensorflow/python/profiler/tfprof_logger.py 5sz-_fill_missing_graph_shape..z!Node %s incompatible shapes: %s. ) step_stats dev_stats node_statsoutputget_operation_by_name node_nameKeyErrorlenoutputs enumerate get_shapeis_fully_definedtensor_descriptionshapedimr TensorShape set_shape merge_with ValueErrorsysstderrwrite) graphrun_metadev_stat node_statopei node_stat_outnode_stat_dimsnode_stat_shaper r r_fill_missing_graph_shape"s@      r1cCs(||d}|durt|}|||<|S)zMaps string to id.N)getr)s str_to_idnumr r r_str_id?s r6Tc Cs|rt||}d}i}i}t||d<|D]y}z t||jt}Wnty2|d7}d}Ynwt } |j | _ d} |rJ|j rJt |j | _ d} |r|jr|jD]4\} } } }| jj}| ret| |nd|_| rl| nd|_| rvt| |nd|_|rt||nd|_d|_qRd} | r| || j <q|r|tjjD]'}|jj |vrt } |jj | _ | jt| || j <q||jj jtq|dkr|stj d|||fS) aExtract trainable model parameters and FLOPs for ops from a Graph. Args: graph: tf.Graph. run_meta: RunMetadata proto used to complete shape information. add_trace: Whether to add op trace information. add_trainable_var: Whether to assign tf.compat.v1.trainable_variables() op type '_trainable_variables'. Returns: logged_ops: dict mapping from op_name to OpLogEntry. string_to_id: dict mapping from string to id. rnoneNFTz0%d ops no flops stats due to incomplete shapes. )!r1rget_operationsrget_stats_for_node_defnode_defREGISTERED_FLOP_STATSr#r OpLogEntrynamevalueint float_ops tracebackcode_deftracesaddr6file_idlineno function_idline_idfunc_start_lineget_collection GraphKeysTRAINABLE_VARIABLESr+typesappendr$r%r&)r'r( add_traceadd_trainable_varop_missing_shape logged_ops string_to_idr+statsentry add_entryfilenamerHfuncnamelinetracevr r r_get_logged_opsHs^             r^c Cs|s ts t}t}|s|St||||d\}}|s'|j| nUi}|jD]} | || j <q,| D];\} } | |vro|| j | j | j dkr[|| j dkr[| j || _ | jjrn|| jjsn|| j| jq8| || <q8|j| | D] \} } | |j| <q|S)aMerge the tfprof default extra info with caller's op_log. Args: graph: tf.Graph. If None and eager execution is not enabled, use default graph. op_log: OpLogProto proto. run_meta: RunMetadata proto used to complete shape information. add_trace: Whether to add op trace information. add_trainable_var: Whether to assign tf.compat.v1.trainable_variables() op type '_trainable_variables'. Returns: tmp_op_log: Merged OpLogProto proto. )rQrRr)rexecuting_eagerlyrget_default_graphr OpLogProtor^ log_entriesextendvaluesr?itemsrOrBrDrE MergeFrom id_to_string) r'op_logr(rQrR tmp_op_logrTrUall_opsrWop_namer3r-r r rmerge_default_with_oplogs4       rlzprofiler.write_op_log)v1cCsl|s ts t}t||||}ttj |dd}| | WddS1s/wYdS)aLog provided 'op_log', and add additional model information below. The API also assigns ops in tf.compat.v1.trainable_variables() an op type called '_trainable_variables'. The API also logs 'flops' statistics for ops with op.RegisterStatistics() defined. flops calculation depends on Tensor shapes defined in 'graph', which might not be complete. 'run_meta', if provided, completes the shape information with best effort. Args: graph: tf.Graph. If None and eager execution is not enabled, use default graph. log_dir: directory to write the log file. op_log: (Optional) OpLogProto proto to be written. If not provided, an new one is created. run_meta: (Optional) RunMetadata proto that helps flops computation using run time shape information. add_trace: Whether to add python code trace information. Used to support "code" view. tfprof_logwN) rr_rr`rlrOpenospathjoinr&SerializeToString)r'log_dirrhr(rQlogr r r write_op_logs "rw)NTT)NNTT)NNT)__doc__rqr$tensorflow.core.profilerrtensorflow.python.eagerrtensorflow.python.frameworkrrtensorflow.python.platformr#tensorflow.python.profiler.internalr tensorflow.python.util.tf_exportrrNr=r1r6r^rlrwr r r rs,         E /