o 2h_"@sbddlmZddlmZddlmZddlmZedgGdddejZej dej e_ d S) ) initializers)ops) keras_export) optimizerzkeras.optimizers.FtrlcsdeZdZdZ                 dfd d Zfd d ZddZfddZZS)Ftrla Optimizer that implements the FTRL algorithm. "Follow The Regularized Leader" (FTRL) is an optimization algorithm developed at Google for click-through rate prediction in the early 2010s. It is most suitable for shallow models with large and sparse feature spaces. The algorithm is described by [McMahan et al., 2013](https://research.google.com/pubs/archive/41159.pdf). The Keras version has support for both online L2 regularization (the L2 regularization described in the paper above) and shrinkage-type L2 regularization (which is the addition of an L2 penalty to the loss function). Initialization: ```python n = 0 sigma = 0 z = 0 ``` Update rule for one variable `w`: ```python prev_n = n n = n + g ** 2 sigma = (n ** -lr_power - prev_n ** -lr_power) / lr z = z + g - sigma * w if abs(z) < lambda_1: w = 0 else: w = (sgn(z) * lambda_1 - z) / ((beta + sqrt(n)) / alpha + lambda_2) ``` Notation: - `lr` is the learning rate - `g` is the gradient for the variable - `lambda_1` is the L1 regularization strength - `lambda_2` is the L2 regularization strength - `lr_power` is the power to scale n. Check the documentation for the `l2_shrinkage_regularization_strength` parameter for more details when shrinkage is enabled, in which case gradient is replaced with a gradient with shrinkage. Args: learning_rate: A float, a `keras.optimizers.schedules.LearningRateSchedule` instance, or a callable that takes no arguments and returns the actual value to use. The learning rate. Defaults to `0.001`. learning_rate_power: A float value, must be less or equal to zero. Controls how the learning rate decreases during training. Use zero for a fixed learning rate. initial_accumulator_value: The starting value for accumulators. Only zero or positive values are allowed. l1_regularization_strength: A float value, must be greater than or equal to zero. Defaults to `0.0`. l2_regularization_strength: A float value, must be greater than or equal to zero. Defaults to `0.0`. l2_shrinkage_regularization_strength: A float value, must be greater than or equal to zero. This differs from L2 above in that the L2 above is a stabilization penalty, whereas this L2 shrinkage is a magnitude penalty. When input is sparse shrinkage will only happen on the active weights. beta: A float value, representing the beta value from the paper. Defaults to `0.0`. {{base_optimizer_keyword_args}} MbP?皙?NFGz?ftrlc stjd |||| | | | | |||d ||dkr!td|d|dkr-td|d|dkr9td|d|dkrEtd|d|dkrQtd|d||_||_||_||_||_||_dS) N) learning_ratename weight_decayclipnorm clipvalueglobal_clipnormuse_ema ema_momentumema_overwrite_frequencyloss_scale_factorgradient_accumulation_stepsr z^`initial_accumulator_value` needs to be positive or zero. Received: initial_accumulator_value=.zR`learning_rate_power` needs to be negative or zero. Received: learning_rate_power=z``l1_regularization_strength` needs to be positive or zero. Received: l1_regularization_strength=z``l2_regularization_strength` needs to be positive or zero. Received: l2_regularization_strength=zt`l2_shrinkage_regularization_strength` needs to be positive or zero. Received: l2_shrinkage_regularization_strength=) super__init__ ValueErrorlearning_rate_powerinitial_accumulator_valuel1_regularization_strengthl2_regularization_strength$l2_shrinkage_regularization_strengthbeta)selfr rrrr r!r"rrrrrrrrrrkwargs __class__rT/var/www/html/chatgem/venv/lib/python3.10/site-packages/keras/src/optimizers/ftrl.pyrNsh  z Ftrl.__init__csF|jrdSt|t|j}|j|ddg|dgd\|_|_dS)zInitialize optimizer variables. Args: var_list: list of model variables to build Ftrl variables on. N accumulatorlinearzeros) initializer) builtrbuildrConstantradd_optimizer_variables _accumulators_linears)r#var_listaccumulator_initializerr%rr'r-s z Ftrl.buildc Cs(t||j}t||j}|j||}|j||}|j}|j}||jd|}t |t d|j |} t |t |} | |t| t ttt| | t|| ||t tt| | |d|} t||j |j} ||tt| || ||| dS)z=Update step given gradient and the associated model variable.g@N)rcastdtyper0_get_variable_indexr1rr r"addmultiplyr!square assign_addsubtractdividepowercliprassign) r#gradientvariabler lraccumr)lr_powerl2_reg grad_to_use new_accum quadraticlinear_clippedrrr' update_stepsV   zFtrl.update_stepc s2t}||j|j|j|j|j|jd|S)N)rrrr r!r") r get_configupdaterrrr r!r")r#configr%rr'rLs  zFtrl.get_config)rrr r r r r NNNNFr NNNr ) __name__ __module__ __qualname____doc__rr-rKrL __classcell__rrr%r'rs.G K2rz{{base_optimizer_keyword_args}}N) keras.srcrrkeras.src.api_exportrkeras.src.optimizersr OptimizerrrRreplacebase_optimizer_keyword_argsrrrr's   f