Connectionist Temporal Classification

Collapse alignment paths and sum their probabilities without needing presegmented labels.

Connectionist Temporal Classification

Connectionist Temporal Classification (CTC) is a scoring function used to train sequence-to-sequence models, such as recurrent neural networks for speech recognition, without requiring pre-segmented training data. It introduces a special "blank" symbol and allows identical consecutive labels to be merged.

Worked Example

Suppose a sequence has three frames, and at each frame, the probabilities for "blank", "A", and "B" are all 1/3 (Uniform three). If our target is "A", we want to compute the total probability of all alignment paths that collapse to "A".

There are 3^3 = 27 possible alignment paths. The ones that collapse to "A" include paths like:

Blank is denoted by "-". "A - -" collapses to "A". "- A -" collapses to "A". "- - A" collapses to "A". "A A -" collapses to "A". "- A A" collapses to "A". "A A A" collapses to "A".

So there are exactly 6 paths that collapse to "A". Since each path has a probability of (1/3)^3 = 1/27, the total probability of target "A" is 6 * 1/27 = 6/27 = 0.2222222222222222. The positive alignment count is 6.

Experiment

Let's explore how the CTC probability changes with target length:

First, ensure the Frames is set to "Uniform two".
Next, select the Target to be "AA".
Observe that the target probability drops to exactly 0, and the Negative Log Likelihood becomes Infinite, showing that some targets are impossible to emit in fewer frames.

You can try this interactive module on the main site at: https://riano.app/module/connectionist-temporal-classification.

Limitations

This demonstration calculates the exact probability of every single path (up to 3 frames, giving 27 paths). In real applications where sequences can be hundreds or thousands of frames long, calculating all paths is computationally impossible. Instead, the forward-backward algorithm (dynamic programming) is used to calculate the path sum efficiently. Also, this demo uses frozen static frame probabilities instead of an actual trained neural network output sequence.

References

Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks

This original 2006 paper introduces the CTC objective function for aligning input and output sequences in neural networks without requiring explicit alignment during training.

Connectionist Temporal Classification