o 2h*@sddlZddlmZ        d!ddZ     d"ddZd d Zd d Zd dZddZ d#ddZ ddZ ddZ ddZ ddZ d$ddZ    d%ddZdd ZdS)&N)treeFTc * spjddd} |st| t} | djd} | dur,t| ddn| | D]}|jdq0|dur^|jtjkrJt|tj}t |jdkrXtj |dd }|s^| |}durdgd2d d |r| srt d t }g}g}fd d}t rt| n|f  fdd}|durIt|}r|t| D]l}||}|| |t |t \}} |}|st|}n|d}t|||}t|}t|}t fdd|D}t ddt|||D}t||}r||||q|g}|g}q|d}|d}t|}rGt|d||t|}t||dd|t|}nzt| D])}||}|t |t \}}ro||||qM|g}|g}qM|d}|d}t|}n=t }t fddt| Dt fddt| Dtdd| D} | t t \}!}"rˆnd t fddtt|!D}#tjdddd}$dur}%nt}%fdd|%dd d!}&|durrt|dg}tjtjd"d# | fd$d% fd&d'n-ttjrIr>tjdd }'t|'dfd(d% nfd)d% d*d'nd durt d+dt|!D f d,d-}(tjd3|(|$|#f|d.|&})|)dd}n fd/d-}(tjd3|(|$|#f|d.|&})|)dd}|)d}#t d0d|#D}t d1d|D}t|!|}t|!|}|st| |}|||fS)4aS Iterates over the time dimension of a tensor. Args: step_function: RNN step function. Args; `input`; Tensor with shape `(samples, ...)` (no time dimension), representing input for the batch of samples at a certain time step. `states`; List of tensors. Returns; `output`; Tensor with shape `(samples, output_dim)` (no time dimension). `new_states`; List of tensors, same length and shapes as 'states'. The first state in the list must be the output tensor at the previous timestep. inputs: Tensor of temporal data of shape `(samples, time, ...)` (at least 3D), or nested tensors, and each of which has shape `(samples, time, ...)`. initial_states: Tensor with shape `(samples, state_size)` (no time dimension), containing the initial values for the states used in the step function. In the case that state_size is in a nested shape, the shape of initial_states will also follow the nested structure. go_backwards: Boolean. If `True`, do the iteration over the time dimension in reverse order and return the reversed sequence. mask: Binary tensor with shape `(samples, time, 1)`, with a zero for every element that is masked. constants: List of constant values passed at each step. unroll: Whether to unroll the RNN or to use a symbolic `while_loop`. input_length: An integer or a 1-D Tensor, depending on whether the time dimension is fixed-length or not. In case of variable length input, it is used for masking in case there's no mask specified. time_major: Boolean. If `True`, the inputs and outputs will be in shape `(timesteps, batch, ...)`, whereas in the False case, it will be `(batch, timesteps, ...)`. Using `time_major = True` is a bit more efficient because it avoids transposes at the beginning and end of the RNN calculation. However, most TensorFlow data is batch-major, so by default this function accepts input and emits output in batch-major form. zero_output_for_mask: Boolean. If `True`, the output for masked timestep will be zeros, whereas in the `False` case, output from previous timestep is returned. return_all_outputs: Boolean. If `True`, return the recurrent outputs for all timesteps in the sequence. If `False`, only return the output for the last timestep (which consumes less memory). Returns: A tuple, `(last_output, outputs, new_states)`. - `last_output`: the latest output of the rnn, with shape `(samples, ...)`. - `outputs`: - If `return_all_outputs=True`: a tensor with shape `(samples, time, ...)` where each entry `outputs[s, t]` is the output of the step function at time `t` for sample `s` - Else, a tensor equal to `last_output` with shape `(samples, 1, ...)` - `new_states`: list of tensors, latest states returned by the step function, of shape `(samples, ...)`. cSs.ttt|j}d\|d<|d<t||S)Nrrrr)listrangelenshapetf transpose)input_taxesr [/var/www/html/chatgem/venv/lib/python3.10/site-packages/keras/src/backend/tensorflow/rnn.pyswap_batch_timestepQs z rnn..swap_batch_timesteprNaxiscSst|r td|t|rtd|t|jt|j}t|D]}t|d}q&dg||j|d}t ||S)Nz)mask_t is expected to be tensor, but got z*input_t is expected to be tensor, but got rr) r is_nested ValueErrorrrrr expand_dimsas_listtile)mask_tr fixed_dim rank_diff_ multiplesr r r _expand_maskts    zrnn.._expand_maskz/Unrolling requires a fixed number of timesteps.cst|}r ||SN)r unstackreverse)r  go_backwardsr r_process_single_input_ts z$rnn.._process_single_input_tcsfddD}t|S)Ncsg|]}|qSr r ).0t_timer r z2rnn.._get_input_tensor..)rpack_sequence_as)r)inp)inputsprocessed_inputr(r_get_input_tensors zrnn.._get_input_tensorc3s|]}|VqdSr r )r&srrr r   zrnn..cs$|] \}}}t|||VqdSr r where)r&mr1psr r rr3  rc3s,|]\}}tj|jd|dVqdS) input_ta_dtypesizetensor_array_nameN)r TensorArrayr>)r&ir- time_steps_tr rr3s c3s6|]\}}s ||n |t|dgVqdS)rN)r!r r")r&tainput_r#r rr3s  cSsg|]}|dqS)rr )r&r-r r rr*r+zrnn..c3s0|]\}}tj|j|jd|dVqdS) output_ta_)r>r? element_shaper@N)r rAr>r)r&rBout)output_ta_sizer rr3 s int32r))r>namecs|kSr r )r)rrCr rszrnn.. T)condmaximum_iterationsparallel_iterations swap_memorymask_tar=cs |Sr readr()rSr r masking_fn+s zrnn..masking_fncs2tfdd|D}tddt|||DS)Nc3s$|] }|tjdVqdS)r;N)rrr&or2r rr3/s  5rnn..compute_masked_output..csr5r r6)r&r8rXfmr r rr33r:tuplezip)rflat_out flat_mask tiled_mask_t)rrrcompute_masked_output.s  z"rnn..compute_masked_outputc t|Sr )r lessr()rev_input_lengthr rrV= crcr )r greaterr() input_lengthr rrVBrfcstfddt||DS)Nc3s"|] \}}t||VqdSr r6)r&rXzorar rr3Fs  rYr[)rr^r_r rarrbEs css|]}t|VqdSr )r zeros_likerWr r rr3Qr4cstfddD}t|}} |t|t\}}t|} r+nt|} ||| } t|} t|} || | } t|| }rPndtfddt|| D}d|t| ft|S)asRNN step function. Args: time: Current timestep value. output_ta_t: TensorArray. prev_output: tuple of outputs from time - 1. *states: List of states. Returns: Tuple: `(time + 1, output_ta_t, output) + tuple(new_states)` c3|]}|VqdSr rTr&rEr(r rr3a%rnn.._step..rc3 |] \}}||VqdSr writer&rErIta_index_to_writer rr3|  rr\rr,flattenr])r) output_ta_t prev_outputstates current_inputroutput new_states flat_outputflat_mask_outputflat_new_output flat_stateflat_new_stateflat_final_state) rb constantsflat_zero_outputinput_tar.rVreturn_all_outputs step_functionzero_output_for_maskrtr)r_stepUs6       zrnn.._step)body loop_varscstfddD}t|}|t|t\}}t|}t|}r,ndtfddt||D}t|}d|ft|S)a)RNN step function. Args: time: Current timestep value. output_ta_t: TensorArray. *states: List of states. Returns: Tuple: `(time + 1,output_ta_t) + tuple(new_states)` c3rkr rTrlr(r rr3rmrnrc3ror rprrrsr rr3rurrv)r)rxrzr{r|r}rr~)rinitial_statesrr.rrrrrs     css|]}|VqdSr )stackrWr r rr3css|]}|dVqdS)rNr rWr r rr3r)rr )rr map_structurerwr with_rank_at_leastr>boolcastrrrr\rr!r"rrjr7r]r,appendr enumerateconstant reduce_maxrA isinstanceTensorsubtract while_loop)*rr.rr$maskrunrollrh time_majorrrrflattened_inputs time_stepsrFrzsuccessive_statessuccessive_outputsr%r0 mask_listrBr-r|r}r`ry flat_statesflat_new_statesflat_final_states last_outputoutputsinput_time_zerooutput_time_zeror output_tar)max_iterationswhile_loop_kwargsmax_lenr final_outputsr )rrbrrr$rrhrr.rrSrVrJr/rrerrDrrrnnsFI                                      0       rc  Cst||| |du| d} | stddlm}t||r|j}t||r%|j}t||r-|j}z t||||||| | | WStjj yEttjj yOtw)N)use_bias reset_afterrVariable) cudnn_okNotImplementedErrorkeras.src.backend.tensorflowrrvalue _cudnn_grur errorsInvalidArgumentError NotFoundError)r. initial_staterkernelrecurrent_kernelbias activationrecurrent_activationreturn_sequencesr$rrrcudnn_supportedrr r rgrusB     rcCsNddlm}ddlm}||jtj|jfvo&||jtj|jfvo&| o&|o&|SNr) activations)ops keras.srcrrtanhr sigmoid)rrrrrrrr r r_do_gru_arguments_support_cudnns  rcCsJddlm}ddlm}||jtj|jfvo$||jtj|jfvo$| o$|Srr)rrrrrrr r r _do_lstm_arguments_support_cudnn s  rc Cs$ttjttj|ddddS)Nrr>rr)r reduce_any reduce_all logical_notr)rr r r_has_fully_masked_sequencesrcCs,ttt|t|}t|dgdS)NaYou are passing a RNN mask that does not correspond to right-padded sequences, while using cuDNN, which is not supported. With cuDNN, RNN masks can only be used for right-padding, e.g. `[[True, True, False, False]]` would be a valid mask, but any mask that isn't just contiguous `True`'s on the left and contiguous `False`'s on the right would be invalid. You can pass `use_cudnn=False` to your RNN layer to stop using cuDNN (this may be slower).)r logical_andrr_is_sequence_right_paddedAssert)rvalidr r r_assert_valid_mask(s rcsDfddfdd|D}fdd|D}tj||ddS)aUtility function convert variable to cuDNN compatible parameter. Note that Keras weights for kernels are different from the cuDNN format. Eg.: ``` Keras cuDNN [[0, 1, 2], <---> [[0, 2, 4], [3, 4, 5]] [1, 3, 5]] ``` If the input weights need to be in a unified format, then set `transpose_weights=True` to convert the weights. Args: weights: list of weights for the kernels and recurrent kernels. biases: list of biases for individual gate. shape: the shape for the converted variables that will be feed to cuDNN. transpose_weights: boolean, whether to transpose the weights. Returns: The converted weights that can be feed to cuDNN ops as param. csrt|S|Sr )r r )w)transpose_weightsr rconvertWsz+_standardize_cudnn_weights..convertcsg|] }t|qSr r reshaper&x)rrr rr*Zsz._standardize_cudnn_weights..csg|]}t|qSr rr)rr rr*[srr)r concatweightsbiasesrrr )rrrr_standardize_cudnn_weights>s rc CsZt|d}tjt|tjdd}tj||d}tttj|ddtj|ddS)aJCheck the mask tensor and see if it right padded. cuDNN uses the sequence length param to skip the tailing timestep. If the data is left padded, or not a strict right padding (has masked value in the middle of the sequence), then cuDNN won't work properly in those cases. Left padded data: [[False, False, True, True, True]]. Right padded data: [[True, True, True, False, False]]. Mixture of mask/unmasked data: [[True, False, True, False, False]]. Note that for the mixed data example above, the actually data RNN should see are those 2 Trues (index 0 and 2), the index 1 False should be ignored and not pollute the internal states. Args: mask: the Boolean tensor with shape [batch, timestep] Returns: boolean scalar tensor, whether the mask is strictly right padded. rr)maxlenrr)r r reduce_sumrrK sequence_maskrequal)rmax_seq_length count_of_trueright_padded_maskr r rr_s  rcCs$|rdnd}tjt|tj|dS)a^Calculate the sequence length tensor (1-D) based on the masking tensor. The masking tensor is a 2D boolean tensor with shape [batch, timestep]. For any timestep that should be masked, the corresponding field will be False. Consider the following example: a = [[True, True, False, False], [True, True, True, False]] It is a (2, 4) tensor, and the corresponding sequence length result should be 1D tensor with value [2, 3]. Note that the masking tensor must be right padded that could be checked by, e.g., `is_sequence_right_padded()`. Args: mask: Boolean tensor with shape [batch, timestep] or [timestep, batch] if time_major=True. time_major: Boolean, which indicates whether the mask is time major or batch major. Returns: sequence_length: 1D int32 tensor. rrr)r rrrK)rrtimestep_indexr r r"_compute_sequence_length_from_masks rcCsttjdS)NGPU)rr configlist_logical_devicesr r r r_is_gpu_availablesrc  CsD|durt|t||} n&|rt|d} t|d} nt|d} t|d} t| g| } |sF| durFtj|dd}d\} } n|rJdnd\} } tj|| d}tj|d dd}|tj|d dd7}tt|d gd }tj d r|d|d|d<|d<|d |d |d <|d <|d|d|d<|d<|d |d |d <|d <t ||t d gdd}|rtj || | | d}tjj||d|dd| |d\}}}}}|rtj || | | d}tj|| gd}|d }|s| dur|rtj|gdd}tj|| d}| dur |}|stj||rdndd}|||gfS)z>GRU with cuDNN implementation which is only available for GPU.Nrrrrrpermrrrrrr is_cuda_buildTrseq_axis batch_axisrinputinput_hinput_cparams is_trainingrnn_modesequence_lengthsr)rrr rfillr rsplitr sysconfigget_build_inforrreverse_sequenceraw_ops CudnnRNNV3r"squeeze)r.rrrrrrr$rr batch_dimmax_sequence_lengthrrinit_hrrrhrrstater r rrs       rcCs6|dur t||||d}n t|||||d}|otS)N)rrrr)rrrrr)rrr)rrrrrargs_supportedr r rrs rc  Cst||| |dud} | stddlm}t||r|j}t||r$|j}t||r,|j}zt|||||||| | | WStjj yEttjj yOtw)N)rrr) rrrrrr _cudnn_lstmr rrr)r.initial_state_hinitial_state_crrrrrrrr$rrrrr r rlstm*s<      rc  s6|durt|t||} n&|rt|d} t|d} nt|d} t|d} t| g| } |sF| durFtj|dd}d\} }n|rJdnd\} }tj|| d}tj|| d}tj|dddtj|ddd7tt ||fdtj d rfd d d Dtjd ddfdd d Dt td t dgdd}|rtj|| | |d}tjj||||dd| |d\}}}}}|rtj|| | |d}tj|| gd}|d}|s| dur| rtj|gdd}tj|| d}tj|| d}| dur|}| stj||rdndd}||||gfS)Nrrrrrrrr is_rocm_buildcg|]}|qSr r r)rr rr*r+z_cudnn_lstm..)rrrrrrcrr r r) full_biasr rr*r+rTrrrr)rrr rrr rrrrjrrrrr r r r"r )r.rrrrrrrr$rrr rrrrinit_crrrcrrr )rrrr]s~        r)FNNFNFFT)FFFFT)Fr )FFFF) tensorflowr rrrrrrrrrrrrrrrrr r r rsJ  G 6 !!y # 3