GAN Optimization: Rotations, Fixed Points, and a Rank-One Preconditioner
Before you start: Newton, Gauss–Newton, and damping.
In a GAN, one player improves a generator while another improves a discriminator. A better move for one changes the other’s landscape. The combined update is a game vector field, which can rotate even when a familiar scalar loss picture suggests that optimization should settle.
A two-variable game that exposes the problem
Consider $\min_x\max_y f(x,y)$ with $f(x,y)=xy$. The game gradient is $v(x,y)=(\partial_xf,-\partial_yf)=(y,-x)$. Simultaneous descent–ascent updates $p_{t+1}=p_t-\eta v(p_t)$, so $x^\prime=x-\eta y$ and $y^\prime=y+\eta x$.
Every nonzero point moves farther from the equilibrium under this simultaneous update for any positive η. The issue is rotational dynamics, not simply a learning rate that was slightly too large. Alternating updates, extragradient methods, and game-aware preconditioners change the dynamics in different ways.
The specific rank-one construction
The paper builds on curvature ideas but uses a particular gradient outer-product preconditioner. In the paper’s notation, B=λI+vvᵀ and the fixed-point map is p′=p+h(B⁻¹−I)v. Preserve the sign convention: v stacks the minimizing gradient and the negative maximizing gradient. This is not the standard least-squares JᵀJ update from the previous chapter.
Sherman–Morrison gives this inverse algebraically. Multiplying it by v avoids constructing the matrix entirely: B⁻¹v=v/(λ+‖v‖²). The displayed basic update therefore simplifies to a scalar multiple of the game gradient.
Calculate the preconditioned direction
Take v=[1,2] and λ=0.5. Its squared norm is 5, so B⁻¹v=v/5.5 and Δ≈−0.81818v. With h=0.1, the parameter change is approximately [−0.081818,−0.163636]. The matrix form looks expensive, but this basic direction requires only a squared norm and vector scaling.
import numpy as np
def paper_basic_direction(v, damping):
v = np.asarray(v, dtype=float)
if damping <= 0:
raise ValueError("damping must be positive")
return (1 / (damping + v @ v) - 1) * v
v = np.array([1., 2.])
B = 0.5 * np.eye(2) + np.outer(v, v)
assert np.allclose(paper_basic_direction(v, 0.5),
np.linalg.solve(B, v) - v)What the simplification tells us
The scalar can change sign when λ+‖v‖² crosses 1. Do not silently replace it with an always-positive learning-rate multiplier. On the pure bilinear example, any nonzero real scalar step along the rotational field still expands the squared radius by a factor of the form 1+c². This basic formula therefore should not be presented as a universal cure for rotational games.
The paper also presents an adaptive variant with additional moment scaling and discusses local fixed-point conditions. Those assumptions and the exact variant matter. Empirical success on particular GAN architectures does not establish global convergence for arbitrary games. A reproduction should match the specified algorithm, objective, damping, momentum, and signs before comparing results.
Evaluate the joint system
Begin with a deterministic bilinear game and a small nonlinear toy problem where trajectories can be plotted. Then test a GAN with fixed architecture and data, comparing sample quality, diversity, training stability, and wall-clock cost. Inception-based scores are measurements under a chosen evaluator; report seeds, sample counts, preprocessing, and complementary diagnostics rather than treating one score as a complete picture.
Use vector-Jacobian or Jacobian-vector checks on tiny cases to catch sign mistakes. Compare the optimized rank-one implementation with an explicit matrix solve at small dimension, as above. This separates an algebraic implementation error from a genuine limitation of the method.
Check your understanding: Does a rank-one matrix expression necessarily require a dense matrix inverse?
No. Sherman–Morrison reduces this one to vector operations, and multiplying by its own defining vector simplifies further. Inspect the algebra before allocating a P×P matrix.
Sources and further reading
- Mishra et al.: a Gauss–Newton approach for min–max optimization in GANs — The B matrix, fixed-point map, and adaptive variant studied here.
- Goodfellow et al.: Generative Adversarial Nets — The original adversarial learning objective.