Quantile Regression

Worked Example

Quantile regression aims to estimate either the conditional median or other quantiles of a response variable. In this intercept-only model, we find a constant prediction c that minimizes the mean pinball loss for a given quantile tau.

The pinball loss (or check function) for a residual u = y - c is defined as:

rho_tau(u) = tau * u if u >= 0 rho_tau(u) = (tau - 1) * u if u < 0

Consider the Spread dataset [-1, 0, 1, 2, 4] and a target quantile tau = 0.5 (50%). If our constant prediction is c = 1, we compute the residuals and individual losses:

• y_1 = -1 => u_1 = -1 - 1 = -2. Since u_1 < 0, the loss is (0.5 - 1)(-2) = 1.
• y_2 = 0 => u_2 = 0 - 1 = -1. Since u_2 < 0, the loss is (0.5 - 1)(-1) = 0.5.
• y_3 = 1 => u_3 = 1 - 1 = 0. The loss is 0.5(0) = 0.
• y_4 = 2 => u_4 = 2 - 1 = 1. Since u_4 > 0, the loss is 0.5(1) = 0.5.
• y_5 = 4 => u_5 = 4 - 1 = 3. Since u_5 > 0, the loss is 0.5(3) = 1.5.

The mean pinball loss is (1 + 0.5 + 0 + 0.5 + 1.5)/5 = 3.5/5 = 0.7 (or 7/10). At tau = 0.5, each loss is half the absolute residual: the mean absolute error here is 1.4, not 0.7.

The optimal minimizer is determined by the k-th order statistic, where k = 5*tau. Here, k = 5(0.5) = 2.5, which is non-integer, so we take the ceiling: y_(3) = 1. The minimum loss interval is exactly [1, 1], making our prediction c = 1 optimal.

Experiment

You can interact with this model by using the controls on the Quantile Regression interactive module. Try following these steps to see how the optimal interval changes:

1. Set Dataset to Spread, Quantile tau (%) to 50 and Constant prediction to 1. Read mean loss 0.700000 and minimizer interval [1,1].
2. Set Quantile tau (%) to 80. Observe how the positive residuals are penalized more heavily, shifting the optimal prediction interval upwards.
3. Set Constant prediction to 2. The mean pinball loss becomes 14/25, and this prediction falls within the new minimizer interval [2, 4] (since 5(0.8) = 4, an integer, leading to an interval from y_(4) to y_(5)).
4. Move Constant prediction to 4 while tau remains 80. It is still optimal with mean loss 0.560000: the complete minimizer set is an interval, not a single preferred endpoint.
5. Select Dataset = Tied, tau 90 and prediction 1. Every individual loss and the mean loss become zero.

Limitations

This module uses a simplified, intercept-only experiment on a fixed dataset. It does not fit a feature-dependent regression line or predict conditional quantiles for continuous populations. Instead, it is designed solely to illustrate how the asymmetric pinball loss selects specific order statistics or intervals.

Additionally, this is a demonstrative toy model with bounded controls (e.g., 5 data points, fixed tau grid, fixed prediction step) rather than a full production learner capable of gradient-based optimization on arbitrary data.

References

For a complete mathematical breakdown of quantile regression and its implementation using the pinball loss in a production environment, consult the scikit-learn documentation. The metric reference defines the requested quantile through alpha and illustrates asymmetric errors. Substituting tau = 0.5 into the formula above gives half the mean absolute error: this scaling preserves the minimizer but changes the numeric loss. Riano constructs the five-response example independently.

• Scikit-Learn: mean_pinball_loss