o 2hY@stdZddlZddlmZddlmZddlmZddlmZddl m Z ddl m Z dd l m Z dd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZdZdZ e!dgZ"e!gdZ#ededdddd.ddZ$Gdddej%Z&d.d d!Z'd"d#Z(d$d%Z)d&d'Z*ej+d(d)Z,Gd*d+d+e-Z.d,d-Z/dS)/z>xla is an experimental library that provides XLA support APIs.N)xla_ops) xla_ops_grad)attr_value_pb2)summary_op_util)context) def_function)ops) array_ops)control_flow_ops)variable_scope) tf_logging)compat)nest) tf_inspect)collections_abc) deprecated) tf_export_xla_compile_id Placeholder) AudioSummaryAudioSummaryV2HistogramSummary ImageSummary MergeSummaryPrint ScalarSummary TensorSummaryTensorSummaryV2zxla.experimental.compilezXxla.experimental.compile is deprecated. Consider using `@tf.function(jit_compile=True)`.T) warn_oncecs,trtjfdd}|StS)aBuilds an operator that compiles and runs `computation` with XLA. NOTE: In eager mode, `computation` will have `@tf.function` semantics. Args: computation: A Python function that builds a computation to apply to the input. If the function takes n inputs, 'inputs' should be a list of n `Tensor`s. `computation` may return a list of `Tensor`s and `Operation`s. `Tensor`s must come before `Operation`s in the returned list. All `Operation`s returned from `computation` will be executed when evaluating any of the returned output tensors. inputs: A list of inputs or `None` (equivalent to an empty list). Each input can be a nested structure containing values that can be converted to `Tensor`s. Note that passing an N-dimension list of compatible values will result in an N-dimension list of scalar `Tensor`s rather than a single Rank-N `Tensor`. If you need a different behavior, convert parts of `inputs` to `Tensor`s with `tf.convert_to_tensor`. Returns: List of `Tensor`s corresponding to the `Tensor`s from the output of `computation` i.e. the same return value as if computation(*inputs) is called directly, with the following exceptions: * None output: a NoOp would be returned with a control dependency on `computation`. * Single value output: a tuple containing the value would be returned. * Operation-only outputs: a NoOp would be returned with a control dependency on `computation`. TODO(b/121383831): Investigate into removing these special cases. Raises: RuntimeError: When eager execution is enabled. Known issues: When a tf.random operation is built with XLA, the implementation doesn't pass the user provided seed to the XLA compiler. As such, the XLA compiler generates a random number and uses it as a seed when compiling the operation. This implementation causes a violation of the Tensorflow defined semantics in two aspects. First, changing the value of the user defined seed doesn't change the numbers generated by the operation. Second, when a seed is not specified, running the program multiple times will generate the same numbers. cs tSN)_compile_internal computationinputsr"]/var/www/html/chatgem/venv/lib/python3.10/site-packages/tensorflow/python/compiler/xla/xla.pyxla_compile_wrapperss z$compile..xla_compile_wrapper)rexecuting_eagerlyrfunctionr!)r$r%r'r"r#r&compile>s 3 r*csxeZdZdZfddZddZdejfddZdejfd d Z d d Z dejfd dZ e ddZ e ddZZS)XLACompileContextaA `ControlFlowContext` for nodes inside an XLA computation cluster. THIS IS ONLY FOR TENSORFLOW INTERNAL IMPLEMENTATION, DO NO USE DIRECTLY. The primary role of `XLACompileContext` is to mark operators inside a xla.compile() computation with attribute "_xla_compile_id=XYZ", where XYZ is a unique name. `ControlFlowContext` is used to perform the annotation since it integrates with Tensorflow constructs like ResourceVariables. For example, if a `ResourceVariable` is constructed inside a xla.compile() block, the `ResourceVariable` implementation can use `with ops.control_dependencies(None)` to build the variable's definition outside the compiled computation. cs0tt|||_t||_g|_||_dS)aBuilds a new XLACompileContext. Args: name: a unique name for the context, used to populate the `_xla_compile_id` attribute. pivot: a pivot node. Nodes in the XLACompileContext that do not have any inputs will have a control dependency on the pivot node. This ensures that nodes are correctly included in any enclosing control flow contexts. N) superr+__init___namer as_bytes_name_as_bytes_unsupported_ops_pivot)selfnamepivot __class__r"r&r-s    zXLACompileContext.__init__cCsh|jr0ddd|jdtD}tdt|j|t|jtkr2tdt|jtdSdSdS)N cSsg|] }d|j|jfqS)z %s (%s))typer4).0opr"r"r& szCXLACompileContext.report_unsupported_operations..z$%d unsupported operations found: %sz... and %d more)r1join_MAX_WARNING_LINESloggingwarninglen)r3op_strr"r"r&report_unsupported_operationss    z/XLACompileContext.report_unsupported_operationsr;cCsxg}g}|jD]'}d}|}|dur!||krd}n|j}|dus|r)||q||q|||||fS)z2Remove any external control dependency on this op.FNT)control_inputs_get_control_flow_context_outer_contextappend_remove_all_control_inputs_add_control_inputs)r3r;internal_control_inputsexternal_control_inputsxis_internal_opctxtr"r"r&_RemoveExternalControlEdgess"    z-XLACompileContext._RemoveExternalControlEdgesc Cs|jtvrtd|j|j|jtvr|j|tdd|j Dr*t d|jt |j j vr7td|j|t tj|jd|j||j|||\}}|j sa|s`||jntt|j D]}|j |}||}||ur~|||qh|rtd|dd |D}| Wdn1swY|!|d d |j"D}|}|dur|j#$||j%}|dus|j%r|j%&|dSdS) zFCreate op in XLACompileContext and notifies outer context recursively.zhOperation of type %s (%s) is not supported in XLA. Execution will fail if this op is used in the graph. css|]}|jjVqdSr )dtype _is_ref_dtyper:rLr"r"r& sz*XLACompileContext.AddOp..zTNon-resource Variables are not supported inside XLA computations (operator name: %s)z?XLA compiled computations cannot be nested, (operator name: %s))sNcSs$g|]}|jrt|jdjqS)r)outputsr identityr;rRr"r"r&r<s z+XLACompileContext.AddOp..cSsg|]}|jqSr"r4rRr"r"r&r<s)'r9_DENYLISTED_OPSr?errorr4_UNSUPPORTED_OPSr1rGanyr%NotImplementedError_XLA_COMPILE_ATTRnode_defattr ValueError _set_attrr AttrValuer0graphprevent_feedingprevent_fetchingrO_add_control_inputr2rangerAAddValue _update_inputrcontrol_dependenciesEnterExitrIrU_valuesupdaterF AddInnerOp) r3r;rJrKindexrLreal_x output_namesrr"r"r&AddOpsj              zXLACompileContext.AddOpcCsl|j|jvr|j|j}|dur|S|S|}|j|j|jr.|j|}|j|j||j|j<|S)zCAdd `val` to the current context and its outer context recursively.N)r4rm_external_valuesgetaddrFrh)r3valresultr"r"r&rh s   zXLACompileContext.AddValuecCs$|||jr|j|dSdSr )rsrFro)r3r;r"r"r&ros zXLACompileContext.AddInnerOpcCsdSr r"r3r"r"r& grad_state!szXLACompileContext.grad_statecCs|r |jSdS)z0Forwards to the enclosing while context, if any.F)GetWhileContext back_propryr"r"r&r|)s zXLACompileContext.back_prop)__name__ __module__ __qualname____doc__r-rCr OperationrOrsrhropropertyrzr| __classcell__r"r"r6r&r+|s  I r+c Cs|durg}t|tjstdt|}dd|D}td}t j |dd}t ||d}z^| d dt |D}tj||d }t}|j}|d t ||} Wdn1sdwY||t| } | r{t| \} } nt| \} } || W||n ||w| st j| d dSd dt | D} t| ddt | D} Wdn1swY| stj| | d } | S)aEBuilds graph operators that compiles and symbolically executes computation. Args: computation: A Python function that builds the computation to compile and execute. inputs: A list of inputs or `None` (equivalent to an empty list). Each input can be a nested structure containing values that are convertible to tensors. Note that passing an N-dimension list of compatible values will result in a N-dimension list of scalar tensors rather than a single Rank-N tensors. If you need different behavior, convert part of inputs to tensors with `tf.convert_to_tensor`. Returns: Same data structure as if computation(*inputs) is called directly with some exceptions for correctness. Exceptions include: 1) None output 2) Single value output 3) Operation-only outputs Raises: ValueError: If any element in computation outputs is neither an operations or a value that can be converted to tensor. ValueError: If computation outputs is non-flat and contains any Operations. TypeError: If `inputs` is not a list or tuple. Nzinputs must be a listcSsg|]}t|qSr")rconvert_to_tensorrRr"r"r&r<Qsz%_compile_internal..clusterz/pivotrW)r4r5cS$g|]\}}tj|d|dqS)zinput_{}rW)r rVformat)r:irLr"r"r&r<[) structure flat_sequenceToutput_0cSr)zoutput{}rW)rxla_cluster_outputrr:ror"r"r&r<rcSs"g|] \}}tj|d|dqS)z output_%drW)r rVrr"r"r&r<s) isinstancerSequence TypeErrorrflattenrget_default_graph unique_namer no_opr+rk enumeratepack_sequence_asr get_variable_scope use_resourceset_use_resource_disable_summary_contextis_flat_postprocess_flat_outputs_postprocess_non_flat_outputs ExitResultrCrlgrouprj) r$r% flat_inputs cluster_namer5rcomputation_inputsvscopesaved_use_resourcerUoutputs_is_flatoutput_tensors control_depsr"r"r&r!1s`            r!cCsdt|tjr |D]}t|tjst|tjst|jdrdSqt|tjr(dSt|jdr0dSdS)auChecks if outputs is a flat structure. Following structures and values are considered flat: 1) None 2) A single object 3) A list or tuple of Tensors/Operations The only structures that this function understands are sequences, dictionaries and types defined using the attrs library. E.g. this means that if outputs contains a single user-defined Object, it is considered to be flat. Errors are raised later on if that Object cannot be converted to a Tensor. Args: outputs: Output from `computation` inside `xla.compile`. Returns: A boolean indicates whether outputs is flat. __attrs_attrs__FT)rrrMappinghasattrr7)rUrr"r"r&rs      rc Cs|durt}t|tjs|f}|tf7}z dd|D}Wnty4}ztdt|d}~wwdd|D}dd|D}|||krMtdg}|D]%}t |j r[|j nd| t |Wdn1sqwYqQ||fS) zValidates flat outputs and adds back device assignments. Args: outputs: Output from `computation` inside `xla.compile`. Returns: Tensors and Operations extracted from outputs. NcSs&g|]}t|tjr |nt|qSr")rrrrr:rr"r"r&r<sz-_postprocess_flat_outputs..oXLA computation function return values must all either be Operations or convertible to Tensors. Got error: "%s"cSsg|] }t|tjr|qSr"rrrrr"r"r&r<cSsg|] }t|tjs|qSr"rrr"r"r&r<rzdXLA computation function must return zero or more Tensor values followed by zero or more Operations.)tuplerrrr r Exceptionr`strrdevicerGr rV)rUeoutput_operationsrnew_output_tensorstr"r"r&rs<   rc Csg}t|D]N}t|tjrtd|jzt|}Wnty1}ztdt |d}~wwt |j r:|j nd| t |Wdn1sPwYq|gfS)aValidates non-flat outputs and adds back device assignments. Args: outputs: Output from `computation` inside `xla.compile`. Returns: Tensors extracted from outputs and an empty list because Operations are not allowed in non-flat outputs.. zxla.compile does not support Operation as return value in non-flat output structure. You can set returned Operations as control dependencies of returned Tensors so Operations are triggered when Tensors are evaluated. Operation found: "%s"rNr)rrrrrr`r4rrrrrGr rV)rUrrrr"r"r&rs.  rccs.tj}ddt_z dVW|t_dS|t_w)aEnters a context where all summary ops are skipped. Summaries are not yet supported in xla.compile(). So we provide this context manager that can skip creating summary ops. This is a temporary workaround due to XLA not supporting summary ops. Yields: None. cSsdS)NTr"r"r"r"r&%sz*_disable_summary_context..N)r skip_summary)original_skip_summary_funcr"r"r&rs  rc@s(eZdZdZddZddZddZdS) _CapturedObjectz#A placeholder to capture an object.cCs d|_dSr _objectryr"r"r&r-0s z_CapturedObject.__init__cCs|jrtd||_dS)NzFInternalError: _CapturedObject can capture only once. Please file bug.)r RuntimeError)r3rr"r"r&capture3s  z_CapturedObject.capturecCs|jSr rryr"r"r&ru;sz_CapturedObject.getN)r}r~rrr-rrur"r"r"r&r-s  rc Csdd}|}|dur||j7}t|}t|j}|jdur!d}nt|j}||}||krA|dkr<|jdur<|d|S|d|S|jdurX||krX|dkrS|d|S|d|SdS)aValidate the number of input arguments to an XLA function. Args: func: the Python function that will be called to generate the body of an XLA computation graph. input_arity: the number of explicit arguments supplied by the caller. infeed_queue: if not None, the infeed queue that will supply additional arguments to the function. Returns: None if function can be called with the supplied number of arguments, or an error string if it cannot. cSsd|||dkr dfSdfS)Nz%s %d argument%srrTr") complaintquantityr"r"r& format_errorMs z3check_function_argument_count..format_errorNrexactlyzat leastzat most)number_of_tuple_elementsr getargspecrAargsdefaultsvarargs) func input_arity infeed_queuernum_args_suppliedarg_spec num_func_argsnum_func_defaults min_func_argsr"r"r&check_function_argument_count?s&         rr )0r contextlibtensorflow.compiler.jit.opsrrtensorflow.core.frameworkrtensorflow.python.distributertensorflow.python.eagerrrtensorflow.python.frameworkrtensorflow.python.opsr r r tensorflow.python.platformr r?tensorflow.python.utilr rrtensorflow.python.util.compatr"tensorflow.python.util.deprecationr tensorflow.python.util.tf_exportrr]r>setrXrZr*XLAControlFlowContextr+r!rrrcontextmanagerrobjectrrr"r"r"r&sR                   9 6g*4#