Positive–Unlabeled Risk Correction

Separate positive and unlabeled loss terms without pretending missing labels are negatives.

Positive–Unlabeled Risk Correction

Concept Overview

In Positive-Unlabeled (PU) learning, we only have access to a set of labeled positive examples and a set of unlabeled examples, which are a mixture of both positive and negative instances. A naive approach treats unlabeled data as negative instances, which biases the model. The unbiased PU learning approach rewrites the expected risk over the true negative distribution using only the positive and unlabeled marginal distributions, based on an assumed positive class prior.

Worked Example: Empirical Estimation

Suppose we have a logistic model with Weight = 2 and Bias = 0. Let's compute the unbiased empirical PU objective for a fixed positive sample P = [1, 2] and an unlabeled sample U = [-2, -1, 0, 1, 2], assuming a positive prior of 0.5.

1. Logistic Losses:

ell_plus(g(x)) = log(1 + exp(-g(x)))
ell_minus(g(x)) = log(1 + exp(g(x)))

2. Sample Computations:

For P = [1, 2], scores are [2, 4].
Average ell_plus over P: ~0.0725
Average ell_minus over P: ~3.0725
For U = [-2, -1, 0, 1, 2], scores are [-4, -2, 0, 2, 4].
Average ell_minus over U: ~1.3967
Estimated Negative Contribution (C):
C = R_U_minus - prior * R_P_minus
C = 1.3967 - 0.5 * 3.0725 = -0.1396
Unbiased PU Risk:
prior * R_P_plus + C = 0.5 * 0.0725 + (-0.1396) = -0.1033
Nonnegative PU Risk:
Since C is negative, we clamp it to 0.
Nonnegative Risk = 0.5 * 0.0725 + 0 = 0.0363

Experiment: Interactive Estimation

Use the Positive–Unlabeled Risk Correction module to explore how the risk estimate changes with different model parameters.

Start with the default settings: Weight = 2, Bias = 0, and Positive prior = 0.5. Observe the unbiased PU risk is negative because the estimated negative contribution drops below zero.
Change the Positive prior to 0.75. Notice how the expected contribution from true positives to the unlabeled negative loss increases, driving the estimated negative contribution even further negative (-0.9077). The Nonnegative PU Risk corrects this by clamping the contribution.
Set Weight = 0 and Bias = 0. The model assigns a score of 0 to all points. Both ell_plus and ell_minus evaluate to log(2) ≈ 0.6931. The unbiased risk exactly equals this constant, and no non-negativity correction is needed.

Limitations

The non-negativity correction addresses an important limitation in practical PU learning with flexible models like neural networks. However, several other considerations apply:

Sampling Assumption: The unbiased risk formulation assumes that the labeled positive sample P is drawn i.i.d. from the true positive distribution, and the unlabeled sample U is drawn i.i.d. from the marginal distribution. Selection bias invalidates this estimator.
Prior Knowledge: The class prior π is assumed to be known or estimated accurately beforehand. In practice, estimating the true class prior from PU data alone is a challenging problem (class prior estimation).
Finite Sample Effects: A finite unbiased estimate may be negative even though the population risk is inherently non-negative. This module illustrates this estimation variance; it does not prove the chosen prior is met by the sample.

References

The theory behind unbiased PU learning and the non-negative risk estimator is primarily detailed in the following literature:

Positive-Unlabeled Learning with Non-Negative Risk Estimator (Kiryo et al., 2017)

Positive–Unlabeled Risk Correction