FGSM Linear Adversarial Examples

Adversarial examples are carefully crafted inputs designed to fool machine learning models. Even simple linear models are susceptible to these attacks. The Fast Gradient Sign Method (FGSM) is a straightforward and computationally efficient way to generate adversarial examples by taking a step in the direction of the gradient of the loss function with respect to the input.

Worked Example: Diagonal Weights

Consider a binary classifier with weights w = [1, 1] and bias b = 0. Let our target point be x = [0.2, 0.1] and the true label be y = 1. The logit is z = w · x + b = 1(0.2) + 1(0.1) = 0.3. The probability of the positive class is p = sigmoid(0.3) ≈ 0.574.

The loss function is the binary cross-entropy, mathematically equivalent to softplus(z) - y * z. The gradient of the loss with respect to the input x is (p - y) * w. Here, p - y ≈ 0.574 - 1 = -0.426. Therefore, the gradient is [-0.426, -0.426].

Using FGSM with a budget (L-infinity norm bound) of ε = 0.25, we perturb the input:δ = ε * sign(gradient). The sign of the gradient is [-1, -1]. So the perturbation is δ = [-0.25, -0.25]. The adversarial example is x_adv = x + δ = [0.2 - 0.25, 0.1 - 0.25] = [-0.05, -0.15].

Original logit: 0.3000
Original probability: 0.5744
Adversarial logit: -0.2000
Adversarial probability: 0.4501

Because the new logit is now negative, the model predicts class 0, and the prediction has been successfully changed!

Experimenting with FGSM

You can interactively observe how FGSM pushes the data point across the decision boundary. Change the Budget to see how a larger perturbation allowance makes it easier to flip the prediction. Adjust the Weight vector and Point to see how the gradient direction and distance to the decision boundary affect the adversarial vulnerability.

Select the Budget control and set it to 0.25.
Select the Weight vector control and change it to observe the gradient shift.
Note how the adversarial point jumps to the corner of the allowed region to maximize the loss.

Open the Interactive Module to experiment with these controls directly. Observe how the adversarial loss is strictly maximized within the L-infinity ball constraint for this linear model.

Limitations

While FGSM is exact and optimal for maximizing the linear approximation of the loss, it has limitations when applied to more complex, highly non-linear models (like deep neural networks). For non-linear models, taking a single large step in the direction of the sign of the gradient might "overshoot" and not actually maximize the true loss within the epsilon-ball.

Additionally, FGSM is a "white-box" attack, meaning it requires full knowledge of the model's architecture and weights to compute the gradient. It does not natively handle "black-box" scenarios where the attacker only has access to the model's outputs.

References

The Fast Gradient Sign Method was originally introduced in the following paper:

Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014).
Explaining and Harnessing Adversarial Examples.
https://arxiv.org/abs/1412.6572

FGSM Linear Adversarial Examples