PCGrad Conflicting Task Gradients

Worked Example

Let's manually trace the PCGrad algorithm for the Conflict case with Scale 1 and Sum aggregation.

1. Identify gradients: Let task 1 gradient be g1 = [1, 0] and task 2 gradient be g2 = [-1, 1].
2. Check for conflict: Compute the dot product between the original gradients.

   d = g1 · g2 = (1)(-1) + (0)(1) = -1.0

   Since d < 0, a conflict is detected. We must project the gradients.
3. Project g1: Project g1 onto the normal of the original g2.

   ||g2||² = (-1)² + 1² = 2 g1_new = g1 - (d / ||g2||²) * g2 = [1, 0] - (-1.0 / 2) * [-1, 1] = [1, 0] + 0.5 * [-1, 1] = [0.5, 0.5]

4. Project g2: Project g2 onto the normal of the original g1.

   ||g1||² = 1² + 0² = 1 g2_new = g2 - (d / ||g1||²) * g1 = [-1, 1] - (-1.0 / 1) * [1, 0] = [-1, 1] + 1 * [1, 0] = [0.0, 1.0]

5. Compute Combined Gradient: Sum the projected gradients.

   H = g1_new + g2_new = [0.5, 0.5] + [0.0, 1.0] = [0.5, 1.5]

6. Expected Results:
   • Original dot product: -1.0
   • Combined gradient x: 0.5
   • Combined gradient y: 1.5

Experiment

You can use the interactive module to observe how different scaling factors and initial gradient alignments affect the final projected descent direction.

1. Open the PCGrad Conflicting Task Gradients module.
2. Set Gradients to "Conflict". You will see the original gradients pointing in opposing directions.
3. Set Second scale to 2. Notice how the magnitude of g2 increases, which changes the projection vectors and the final combined gradient.
4. Change Aggregation to "Mean". The combined gradient will be halved, but the direction remains the same.

Limitations

While PCGrad effectively removes direct interference between conflicting gradients, it has certain limitations:

• Opposed gradients: If two gradients are exactly opposed (e.g., pointing directly at each other, as in the "Opposed" preset), they can both project to zero. In this case, the combined descent direction becomes a zero vector, which halts optimization progress for those parameters in that step.
• Two-task deterministic focus: This interactive module demonstrates the deterministic two-task algorithm. General multi-task PCGrad with three or more tasks requires randomly sampling pairs of conflicting gradients in an iterative process, making the final gradient order-dependent.
• No guarantee of loss decrease: Projecting gradients guarantees that the directional derivative is non-positive for each task locally, but it does not guarantee a finite-step decrease in the global loss function.

References

The PCGrad algorithm was introduced as a method to mitigate negative transfer in multi-task learning by projecting conflicting gradients.

• Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., & Finn, C. (2020). Gradient Surgery for Multi-Task Learning.arXiv:2001.06782.