claudini/claudini/methods/original/egd

name, full_name, reference, paper_url, code

name	full_name	reference	paper_url	code
EGD	Exponentiated Gradient Descent	biswas2025adversarial	https://arxiv.org/abs/2505.09820	https://github.com/sbamit/Exponentiated-Gradient-Descent-LLM-Attack

EGD — Exponentiated Gradient Descent Attack

Paper: Biswas et al., "Adversarial Attack on Large Language Models using Exponentiated Gradient Descent" (2025) Links: arXiv | Code \cite{biswas2025adversarial}

Algorithm

Optimizes probability distributions over the vocabulary for each token position using exponentiated gradient descent (multiplicative updates) with Adam-style momentum. Key difference from PGD: uses param *= exp(-lr * adam_grad) instead of additive param -= lr * adam_grad, which naturally preserves positivity on the simplex. Row normalization after each step ensures valid distributions.

Loss

Combined loss = CE_target - entropy_reg + KL_sharpening

• CE at target positions (standard cross-entropy)
• Entropy regularization (negative entropy, annealed coefficient 1e-5 → 1e-3): encourages sharper distributions over time
• KL sharpening (-log(max(dist, dim=1))): pushes the leading probability higher for better discretization

Key Hyperparameters

Parameter	Default	Description
lr	0.1	EGD learning rate
beta1	0.9	Adam first moment decay
beta2	0.999	Adam second moment decay
eps	1e-4	Adam epsilon
grad_clip	1.0	Max gradient norm
reg_init	1e-5	Initial regularization coefficient
reg_final	1e-3	Final regularization coefficient
scheduler_patience	50	ReduceLROnPlateau patience
scheduler_factor	0.1	ReduceLROnPlateau factor