o 2hoB@s>dZddlZddlZddlZddlZddlZddlmZmZm Z m Z ddl Z ddl m Zddl mZddlmZdZdd Zd d Zd d ZddZd3ddZddZddZddZd4ddZddZddZdd Zd!d"Zd#d$Z d%d&Z!d'd(Z"d)d*Z#d+d,Z$e d-Z%d.e ej&ej'fd/ee%d0ee%fd1d2Z(dS)5zUtility functions for FlatBuffers. All functions that are commonly used to work with FlatBuffers. Refer to the tensorflow lite flatbuffer schema here: tensorflow/lite/schema/schema.fbs N)OptionalTypeTypeVarUnion)schema_py_generated) schema_util)gfilesTFL3cCstj|d}tj|S)zBConverts a tflite model from a bytearray to an object for parsing.r) schema_fbModelGetRootAsModelModelT InitFromObj)model_bytearray model_objectra/var/www/html/chatgem/venv/lib/python3.10/site-packages/tensorflow/lite/tools/flatbuffer_utils.pyconvert_bytearray_to_object's rcCs\t|s td|t|d}t|}Wdt|S1s%wYt|S)a2Reads a tflite model as a python object. Args: input_tflite_file: Full path name to the input tflite file Raises: RuntimeError: If input_tflite_file path is invalid. IOError: If input_tflite_file cannot be opened. Returns: A python object corresponding to the input tflite file. zInput file not found at %r rbN)rExists RuntimeErrorGFile bytearrayreadread_model_from_bytearray)input_tflite_fileinput_file_handlerrrr read_model-s  rcCst|}tjdkrt|dd|jD]}|jr)||j|j|j|_d|_d|_q|jD]}|j D]}|j rI||j |j |j |_ d|_ d|_ q2q-|S)zReads a tflite model as a python object. Args: model_bytearray: TFLite model in bytearray format. Returns: A python object corresponding to the input tflite file. biglittler) rsys byteorderbyte_swap_tflite_model_objbuffersoffsetsizedata subgraphs operatorslargeCustomOptionsOffsetlargeCustomOptionsSize customOptions)rmodelbuffersubgraphoprrrrAs,      rcCstt|S)aCReads a tflite model as a python object with mutable tensors. Similar to read_model() with the addition that the returned object has mutable tensors (read_model() returns an object with immutable tensors). NOTE: This API only works for TFLite generated with _experimental_use_buffer_offset=false Args: input_tflite_file: Full path name to the input tflite file Raises: RuntimeError: If input_tflite_file path is invalid. IOError: If input_tflite_file cannot be opened. Returns: A mutable python object corresponding to the input tflite file. )copydeepcopyr)rrrrread_model_with_mutable_tensorsasr1cCs:td}||}|j|tdt|}||}|S)z@Converts a tflite model from an object to a immutable bytearray.i)file_identifier) flatbuffersBuilderPackFinish_TFLITE_FILE_IDENTIFIERbytesOutput)r extra_bufferbuilder model_offsetrrrrconvert_object_to_bytearrayws   r>cCsdtjdkrt|}t|ddt|}t|d}||WddS1s+wYdS)asWrites the tflite model, a python object, into the output file. NOTE: This API only works for TFLite generated with _experimental_use_buffer_offset=false Args: model_object: A tflite model as a python object output_tflite_file: Full path name to the output tflite file. Raises: IOError: If output_tflite_file path is invalid or cannot be opened. rrwbN) rr r/r0r!r>rrwrite)routput_tflite_fileroutput_file_handlerrr write_models   "rCcCs4d|_|jD]}d|_|jD]}d|_qqd|_dS)ayStrips all nonessential strings from the model to reduce model size. We remove the following strings: (find strings by searching ":string" in the tensorflow lite flatbuffer schema) 1. Model description 2. SubGraph name 3. Tensor names We retain OperatorCode custom_code and Metadata name. Args: model: The model from which to remove nonessential strings. N) descriptionr&nametensors signatureDefs)r+r-tensorrrr strip_stringss   rIcCs*tjjD] \}}||kr|SqdS)z4Converts a numerical enum to a readable tensor type.N)r TensorType__dict__items) tensor_typerEvaluerrr type_to_names rOcs6t||j}tdt|}durfdd|D}i}|jD]"}|jD]}|jdur/n|jD]}|j|} t | j || j <q2q&q!|D]R} || j } | durSdn| j } | dkr[qF|| d} | dr| dkrld nd }td| t|D]}td d }t|| ||qwqFt| D] }tdd | |<qqFdS)aRandomize weights in a model. Args: model: The model in which to randomize weights. random_seed: The input to the random number generator (default value is 0). buffers_to_skip: The list of buffer indices to skip. The weights in these buffers are left unmodified. Ncsg|]}|vr|qSrr).0idxbuffers_to_skiprr z%randomize_weights..rINT8FLOATFLOAT16efgg?)randomseedr"rangelenr&r'inputsrFrOtyper,r%r$get startswithstructcalcsizeuniform pack_intorandint)r+ random_seedrTr" buffer_ids buffer_typesgraphr. input_idxrHi buffer_i_data buffer_i_size buffer_type format_coder#rNjrrSrrandomize_weightss>           rucCs:|jD]}|jr|jd}||vr||d|_qdS)zRename custom ops so they use the same naming style as builtin ops. Args: model: The input tflite model. map_custom_op_renames: A mapping from old to new custom op names. asciiN) operatorCodes customCodedecodeencode)r+map_custom_op_renamesop_code op_code_strrrrrename_custom_opss  r~cCsD|j|}t|j|j}ttjD] \}}||kr|SqdS)zConverts a TFLite op_code to the human readable name. Args: model: The input tflite model. op_code: The op_code to resolve to a readable name. Returns: A string containing the human readable op name, or None if not resolvable. N)rwmax builtinCodedeprecatedBuiltinCodevarsr BuiltinOperatorrL)r+r|r.coderErNrrropcode_to_names rc Cstd}t}t|3}|D]%}||}|durq|d}td|d}dd|D}||qWdt |S1sBwYt |S)a@Converts xxd output C++ source file to bytes (immutable). Args: input_cc_file: Full path name to th C++ source file dumped by xxd Raises: RuntimeError: If input_cc_file path is invalid. IOError: If input_cc_file cannot be opened. Returns: A bytearray corresponding to the input cc file array. z\W*(0x[0-9a-fA-F,x ]+).*NrP,cSsg|]}t|ddqS))base)int)rQxrrrrU*rVz'xxd_output_to_bytes..) recompileropenmatchgroupfiltersplitextendr9) input_cc_filepatternr file_handleline values_match list_text values_textvaluesrrrxxd_output_to_bytes s      rcCst|}t|S)a7Converts xxd output C++ source file to object. Args: input_cc_file: Full path name to th C++ source file dumped by xxd Raises: RuntimeError: If input_cc_file path is invalid. IOError: If input_cc_file cannot be opened. Returns: A python object corresponding to the input tflite file. )rr)r model_bytesrrrxxd_output_to_object0s rcsDfddtdtjD}dfdd|D_dS)z4Helper function for byte-swapping the buffers field.csg|] }j||qSr)r%rQro)r, chunksizerrrUCsz,byte_swap_buffer_content..rr2cs g|] }t|qSr)r from_bytesto_bytes)rQbyteswap)r from_endiness to_endinessrrrUGsN)r_r`r%join)r,rrrto_swapr)r,rrrrbyte_swap_buffer_contentAs rcsntjdd}tjd|dd}dfddtd|ddddD}||_dS) aHelper function for byte-swapping the string buffer. Args: buffer: TFLite string buffer of from_endiness format. from_endiness: The original endianness format of the string buffer. to_endiness: The destined endianness format of the string buffer. rNr2cs.g|]}tj||ddqS)r)rrr%rrr,rrrrrUWs z,byte_swap_string_content..rP)rrr%rrr_)r,rrnum_of_stringsstring_content prefix_datarrrbyte_swap_string_contentMs rc CsH|durdSg}tjjtjjtjjg}tjjtjjtjjtjjg}tjj tjj tjj tjj g}|j D]o}|jD]i}|jdkr|jt|jkr|j|vr|j|jjdur|jtjjkrft|j|j||n4|j|vrwt|j|jd||n#|j|vrt|j|jd||n|j|vrt|j|jd||nq7||jq7q2dS)aByte swaps the buffers field in a TFLite model. Args: model: TFLite model object of from_endiness format. from_endiness: The original endianness format of the buffers in model. to_endiness: The destined endianness format of the buffers in model. Nrrr)r rJrYINT16UINT16FLOAT32INT32 COMPLEX64UINT32INT64FLOAT64 COMPLEX128UINT64r&rFr,r`r"r%rbSTRINGrrappend) r+rrbuffer_swappedtypes_of_16_bitstypes_of_32_bitstypes_of_64_bitsr-rHrrrr!`sX        r!cCs(|durdSt|}t|||t|S)aGenerates a new model byte array after byte swapping its buffers field. Args: tflite_model: TFLite flatbuffer in a byte array. from_endiness: The original endianness format of the buffers in tflite_model. to_endiness: The destined endianness format of the buffers in tflite_model. Returns: TFLite flatbuffer in a byte array, after being byte swapped to to_endiness format. N)rr!r>) tflite_modelrrr+rrrbyte_swap_tflite_buffers   rcCsrt|tjs t|}t}|jD]$}|jdurq|jD]}t|j |j }|tj j kr3| |jjqqt|S)zCalculates the number of unique resource variables in a model. Args: model: the input tflite model, either as bytearray or object. Returns: An integer number representing the number of unique resource variables. N) isinstancer r rsetr&r'r#get_builtin_code_from_operator_coderw opcodeIndexr VAR_HANDLEaddbuiltinOptions sharedNamer`)r+unique_shared_namesr-r. builtin_coderrrcount_resource_variabless      rOptsTr. opts_typereturnc Cstd|}|j}|ds||d}ttj|}|s&ttj|s&|t|tj r`|s@t tj|}tj }|}| } nt tj|}tj }|}|} |dusY| |kr[dS|||St|tjr||rl|j} n|j} | dusxt| |szdS| SdS)aGet the options of an operator as the specified type. Requested type must be an object-api type (ends in 'T'). Args: op: The operator to get the options from. opts_type: The type of the options to get. Returns: The options as the specified type, or None if the options are not of the specified type. Raises: ValueError: If the specified type is not a valid options type. zUnsupported options type: TN) ValueError__name__endswith removesuffixhasattrr BuiltinOptionsBuiltinOptions2rOperatorgetattrBuiltinOptions2CreatorBuiltinOptions2TypeBuiltinOptionsCreatorBuiltinOptionsType OperatorTrbuiltinOptions2) r.rerr type_namebase_type_name is_opt_1_typeenum_val opts_creatorraw_opsactual_enum_val raw_ops_trrrget_options_ass>         r)r2)rN))__doc__r/r]rrertypingrrrrr4tensorflow.lite.pythonrr rtensorflow.python.platformrr8rrrr1r>rCrIrOrur~rrrrrr!rrrrrrrrrrsL      3' :