Spearman Average Ranks and Permutations

Worked Example

Consider the "One tie" scenario with a sample size of 4, where the variables are x = [1, 1, 2, 3] and y = [1, 2, 3, 4].

First, we assign average ONE-based ranks to each variable:

1. For x = [1, 1, 2, 3], the values '1' occupy positions 1 and 2. Their average rank is (1 + 2) / 2 = 1.5. The value '2' gets rank 3, and '3' gets rank 4. So, rankX = [1.5, 1.5, 3, 4].
2. For y = [1, 2, 3, 4], there are no ties. The ranks are simply their positions: rankY = [1, 2, 3, 4].

Next, we calculate the Pearson correlation (Spearman rho) of these ranks.

mean(rankX) = (1.5 + 1.5 + 3 + 4) / 4 = 2.5 mean(rankY) = (1 + 2 + 3 + 4) / 4 = 2.5 Numerator = Σ(rx - mx)(ry - my) = (1.5 - 2.5)(1 - 2.5) + (1.5 - 2.5)(2 - 2.5) + (3 - 2.5)(3 - 2.5) + (4 - 2.5)(4 - 2.5) = (-1)(-1.5) + (-1)(-0.5) + (0.5)(0.5) + (1.5)(1.5) = 1.5 + 0.5 + 0.25 + 2.25 = 4.5 DenominatorX = Σ(rx - mx)² = (-1)² + (-1)² + 0.5² + 1.5² = 1 + 1 + 0.25 + 2.25 = 4.5 DenominatorY = Σ(ry - my)² = (-1.5)² + (-0.5)² + 0.5² + 1.5² = 2.25 + 0.25 + 0.25 + 2.25 = 5.0 rho = 4.5 / sqrt(4.5 * 5.0) = 4.5 / sqrt(22.5) ≈ 0.948683298

The squared rank distance sum(d²) = (1.5-1)² + (1.5-2)² + (3-3)² + (4-4)² = 0.25 + 0.25 + 0 + 0 = 0.5. The no-tie shortcut is invalid here because ties are present.

Experiment

Explore the impact of ties on the Spearman correlation coefficient using the interactive module.

1. Open the module and locate the control labeled "Data". Change its value to "Monotone". Observe that the "Spearman rho" is 1.0 and the "No-tie shortcut" also yields 1.0 because there are no ties.
2. Change the "Data" control to "Joint ties". Notice that the "Ties present" result changes to "Yes" and the "No-tie shortcut" becomes "Unavailable", demonstrating that the shortcut formula 1 - 6 sum(d²)/(n(n²-1)) cannot be used when ties are present.
3. Set the "Sample size" control to 3. Observe how the "Exact two-sided p" value and the graph update to reflect the smaller sample size and the reduced number of permutations.

Limitations

A significant limitation of the Spearman rank correlation is its behavior with constant variables. If one or both variables have zero variance (i.e., all values are identical), the correlation coefficient is undefined because the denominator in the Pearson correlation of the ranks becomes zero. The module appropriately displays "Unavailable" rather than zero for these cases.

Additionally, the common shortcut formula 1 - 6 sum(d²)/(n(n²-1)) is strictly valid only when there are no tied ranks. Applying it to data with ties will yield an incorrect result, which is why the module disables it when ties are detected.

References

For more details on the Spearman rank-order correlation coefficient and the handling of tied ranks, refer to the official SciPy documentation:

• scipy.stats.spearmanr - This source supports the methodology used for calculating average ranks, the Pearson correlation of those ranks, and exact permutation p-values.