Gradient Reversal Domain Adaptation

Domain adaptation aims to align feature representations between a source domain (where labels are available) and a target domain (where labels are not). The Gradient Reversal approach uses an adversarial domain discriminator to distinguish source from target features. During backpropagation, the gradient from this domain classifier is reversed by multiplying it by a negative adversarial weight -λ, encouraging the feature extractor to learn representations that are invariant to the domain shift.

Theory

In a simple linear setup, we map input features x to hidden representations h = w x. From h, two heads compute their respective predictions: a label head (predicting the task label y) with logit v h, and a domain head (predicting the domain d) with logit u h. The label loss L_y minimizes the standard cross-entropy error on source labeled data, while the domain loss L_d minimizes the error on both domains.

To learn domain-invariant features, we want the feature extractor w to minimize L_y but maximize L_d. The gradient reversal layer acts as an identity function during the forward pass. During the backward pass, it scales the gradient by -λ. Therefore, the total feature gradient becomes:

g_w = (∂ L_y)/(∂ w) - λ (∂ L_d)/(∂ w)

Worked Example

Consider the default scenario: feature weight w = 0.5, domain head weight u = 1.0, adversarial weight λ = 1.0, and label head weight v = 1.0. The inputs are X = (-2, -1, 1, 2), with domains D = (0, 1, 0, 1) and labels Y = (0, NaN, 1, NaN).

We first calculate the forward pass and probabilities. The cross-entropy loss is averaged over the respective samples.

Label feature gradient: (∂ L_y)/(∂ w) ≈ -0.4577117557690678
Domain feature gradient: (∂ L_d)/(∂ w) ≈ 0.04228824423093219

Using the gradient reversal layer, the total gradient for the feature weight is:

g_w = -0.4577117557690678 - 1.0 × 0.04228824423093219 = -0.5

With a learning rate of 0.1, the updated feature weight is:

w' = w - 0.1 × g_w = 0.5 - 0.1 × (-0.5) = 0.55

Expected result: the reversed total feature gradient is -0.5, and the updated feature weight is 0.55.

Experiment

Interactive module experiment: Gradient Reversal Domain Adaptation Module.

Follow these steps to observe the adversarial gradient reversal in action:

Observe the baseline gradients with the default values. Note the sign of the reversed total feature gradient.
Change the Adversarial weight to 0. The total feature gradient will no longer subtract the domain gradient, reflecting standard supervised training without domain adaptation.
Change the Feature weight to 0 and the Domain head weight and Adversarial weight to 2. Observe the saddle point behavior.

Limitations

While the gradient reversal layer provides an elegant implementation of adversarial domain adaptation, a single optimization step does not guarantee invariant features. The process involves a delicate minimax game; if the domain discriminator converges too quickly, the gradient for the generator may vanish. Finding an appropriate schedule for λ and the learning rate is crucial in practice to stabilize training.

References

For an in-depth analysis of Domain-Adversarial Training of Neural Networks, refer to:

Domain-Adversarial Training of Neural Networks (Ganin et al., JMLR 2016)

Gradient Reversal Domain Adaptation