Machine Learning

144 articles in this category.

Gradient Descent

Finding the minimum of a cost function.

Perceptron

Visualize the Perceptron algorithm learning a linear decision boundary.

K-Means Clustering

Partitioning data into clusters by iteratively assigning points and updating centroids.

Linear Regression

Fitting a line to data by minimizing the sum of squared residuals.

Attention Mechanism

Attention Mechanism

Decision Boundaries

Visualize how model complexity and regularization shape the separation between classes.

Decision Tree

A non-parametric supervised learning method for classification.

Neural Network Learning

Understanding forward propagation, backpropagation, and weight updates in artificial neural networks.

Support Vector Machine

Visualize how a linear SVM learns a maximum-margin decision boundary using gradient descent on hinge loss.

Backpropagation

Visualize the gradient flow and weight updates in a neural network.

Bias-Variance Tradeoff

Visualize how model complexity affects the tradeoff between bias (underfitting) and variance (overfitting).

Convolutional Filter

Visualize how a kernel slides over an input image to produce a feature map via the convolution operation.

Naive Bayes Classifier

Interactively explore how a Naive Bayes classifier learns probabilities and makes predictions.

Recurrent Neural Network

Recurrent Neural Network.

PCA (Dimensionality Reduction)

Find the principal axes that maximize variance and project data into a lower-dimensional space.

Generative Adversarial Network

Visualize the generator mapping latent noise to a target distribution while the discriminator learns to separate real from fake.

Optimization Landscape

Visualize gradient descent path finding minimums on 2D contour loss surfaces.

Word Embedding (Word2Vec)

Simulate training 2D word vectors using the Skip-gram architecture with Negative Sampling.

Activation Functions

Visualize and compare common neural network activation functions.

Batch Normalization

Visualize the effects of batch normalization on neural network activations.

Confusion Matrix & Metrics

Adjust predictions and true values to see how accuracy, precision, recall, and F1 score react.

Cross-Validation

Visualize K-Fold cross-validation, model fitting, and training versus validation error.

Dropout Regularization

Visualize how randomly disabling neurons during training prevents neural network overfitting.

Ensemble Methods

Combine multiple weak learners to create a single strong model with lower error and greater generalization.

Learning Rate Schedules

Visualize how learning rate decay schedules impact optimization convergence and stability.

Mini-Batch Gradient Descent

Optimize parameters efficiently by estimating gradients on small data batches.

Reinforcement Learning (Q-Learning)

Visualize an agent learning to navigate a grid world using the Q-Learning algorithm.

Transformer Architecture

Visualize the core components and connectivity of a Transformer encoder block.

Autoencoder

Visualize the encoding and decoding process of data through a bottleneck layer.

Data Augmentation Effects

Visualize how image transformations like rotation, scaling, and noise artificially expand training datasets.

Feature Importance (SHAP)

Visualize SHAP values to explain machine learning model predictions based on feature contributions.

Gradient Clipping & Exploding Gradients

Visualize how exploding gradients occur during backpropagation and how gradient clipping mitigates them.

Normalization Techniques

Compare and visualize various data normalization and scaling techniques used in machine learning.

Residual Connections

Visualize how skip connections help mitigate the vanishing gradient problem in deep networks.

t-SNE Visualization

Reduce high-dimensional data into a low-dimensional map by matching pairwise probability distributions.

Contrastive Learning

Visualize how contrastive learning pulls similar examples together and pushes dissimilar ones apart in embedding space.

Knowledge Distillation

Transfer knowledge from a large teacher model to a smaller student model using temperature scaling.

LSTM & Gated Units

Visualize the internal workings of an LSTM cell and its gating mechanisms.

Mixture of Experts

Visualize how a gating network routes inputs to specialized expert networks to form a combined prediction.

Variational Autoencoder

Visualize the encoding of data into a probabilistic latent space and its reconstruction.

Early Stopping

Visualize how monitoring validation loss prevents overfitting by halting training at the optimal epoch.

UMAP Visualization

Reduce high-dimensional data into a low-dimensional map by preserving local topological structures.

Neural Style Transfer

Visualize the process of separating and recombining image content and style using convolutional neural networks.

Probability Calibration & Brier Score

Compare forecast probabilities with observed frequencies and decompose binary Brier error into reliability, resolution and uncertainty.

Hyperparameter Tuning

Search real kNN regressors on fixed training and validation data, replay budgeted grid or seeded trials, and inspect exact score ties and neighbor predictions.

Beam Search Decoding

Compare bounded beam decoding with exhaustive path probabilities, deterministic tie-breaking and zero-probability branches.

Counterfactual Explanations

Find the minimum L1 or Euclidean L2 change to flip a linear classifier decision.

Covariate Shift Importance Weighting

Compute exact two-group risks, density-ratio weights and ESS dispersion, including unsupported target groups.

Expectation-Maximization for Mixtures

Visualize Expectation-Maximization for a two-Gaussian mixture model on one-dimensional data.

Gaussian Process Regression

Gaussian Process Regression with exact function-space Bayesian conditioning and latent uncertainty.

Graph Label Propagation

Semi-supervised harmonic inference with clamped labels on a weighted graph.

Isotonic Regression

Keep x positions in fixed order. Fit nondecreasing theta minimizing weighted sum of squared differences using pool-adjacent-violators algorithm.

Proximal Soft Thresholding

Visualize L1 proximal soft thresholding and compare it with L2 regularization.

Quantile Regression

Fit an intercept-only quantile model on exactly five fixed responses.

Split Conformal Prediction

Finite-sample corrected predictive intervals from held-out residual ranks.

Kullback Leibler Divergence

Compare directional relative entropy for small distributions and see how a missing reference support creates infinity.

Wasserstein Distance One Dimensional

Transport probability mass along a line and compare its minimum travel cost with the area between two CDFs.

Causal Backdoor Adjustment

Separate observational association from an intervention under an explicit causal model.

Direct Preference Relative Policy

Separate response probabilities from preferences through a reference-relative DPO objective.

FGSM Linear Adversarial Examples

Move a toy classifier input within a fixed budget to maximize its actual loss.

Integrated Gradients ReLU Path

Integrate gradients along a baseline path and verify the exact completeness identity.

James–Stein Positive-Part Shrinkage

Inspect radial mean shrinkage while separating realized error from the James–Stein expected-risk theorem.

Linear Attention Reassociation

Reassociate positive-feature attention into compact running sums and expose zero-denominator cases.

Multiclass Temperature Scaling

Adjust confidence without changing a classifier's ranking while tracking stable log loss.

Nesterov Lookahead Momentum

Compare a gradient taken at a momentum lookahead with one taken at the current point.

PCA and ZCA Covariance Whitening

Whiten a covariance while comparing rotated and orientation-preserving views.

AdaGrad Coordinate Learning Rates

Watch accumulated gradients adapt each coordinate’s step size.

Adam Moment Bias Correction

Inspect biased moving moments and the correction that changes Adam's early steps.

Bernoulli Natural Gradient

Compare Euclidean and Fisher-scaled updates without hiding invalid probabilities.

Chow–Liu Dependence Tree

Approximate a joint distribution with the best pairwise dependence tree and expose XOR’s missing interaction.

Connectionist Temporal Classification

Collapse alignment paths and sum their probabilities without needing presegmented labels.

Cost-Complexity Regression Tree Pruning

Compare every rooted subtree and see how a leaf penalty changes the selected regression tree.

DBSCAN Density Clustering

Grow clusters through dense neighborhoods while separating border points from noise.

Elastic Weight Consolidation Toy

Balance a new quadratic task against parameter importance from an old task.

Empirical Rademacher Finite Classes

Enumerate random sign correlations to compare the capacity of finite hypothesis classes.

Finite Neural Tangent Kernel

Measure a network's gradient kernel and test its first-order prediction of an actual training step.

Focal Loss

Down-weight easy examples and compare focal loss with weighted cross-entropy.

Frank–Wolfe Dual Gap

Optimize with a linear vertex oracle and certify progress through a primal–dual gap.

Gaussian Diffusion Forward Process

Watch signal shrink and Gaussian uncertainty grow in a deterministic diffusion experiment.

Gaussian Product of Experts

Contrast Gaussian consensus by multiplication with uncertainty retained by a mixture.

Gradient Reversal Domain Adaptation

Reverse only the domain gradient entering the feature extractor.

Gumbel–Softmax Relaxation

Move a relaxed categorical draw through a simplex as temperature changes.

Huber Loss and Robust Location

Compare bounded Huber influence with the pull of a large squared-error residual.

Jackknife+ Leave-One-Out Intervals

Combine leave-one-out fits and residuals with the finite-sample Jackknife+ quantile correction.

Jensen–Shannon Divergence

Mix two distributions to obtain a bounded divergence with explicit zero-mass conventions.

Kernel Maximum Mean Discrepancy

Compare kernel sample means while distinguishing a squared distance from an unbiased estimator.

Label Shift Prior Correction

Correct a calibrated posterior when class prevalence changes but class-conditional data do not.

Label Smoothing and Cross-Entropy

Mix a one-hot label with uniform mass and inspect the resulting loss and logit gradients.

Linear Discriminant Analysis

Find a class-aware projection by balancing mean separation against within-class covariance.

Locally Weighted Linear Regression

Move a local weighted fit along a curved dataset and inspect the bandwidth tradeoff.

Maximum Entropy Moment Matching

Find the most uncertain distribution consistent with a mean and compare other feasible choices.

Mean Shift Mode Seeking

Follow Gaussian-weighted updates toward stationary points of a density estimate.

Noisy-Label Loss Correction

Correct expected classification risk under a known label-noise channel and expose its singular limit.

PAC–Bayes KL Risk Certificate

Balance empirical errors and posterior complexity with a finite-sample Gibbs-risk certificate.

Polyak Optimal-Value Step

Compare an optimal-value step with exact line search and a fixed learning rate.

Positive–Unlabeled Risk Correction

Separate positive and unlabeled loss terms without pretending missing labels are negatives.

Precision Recall Curves

Move a score threshold and measure precision, recall and stepwise average precision.

Principal Component Regression Shrinkage

Compare discarded principal directions with gradual spectral shrinkage.

Quadratic Discriminant Gaussian Boundary

Derive curved Gaussian class boundaries while preserving priors and covariance normalization.

Random Fourier Kernel Features

Approximate an RBF kernel with fixed random sinusoidal features and inspect finite-sample error.

RANSAC Consensus Line Fitting

Separate consensus inliers from outliers before refitting a line.

RMSNorm Scaling

Normalize feature scale without removing the mean and compare the resulting invariances.

Rotary Position Embeddings

Rotate query and key vectors to make attention depend on relative position.

Signed Feature Hashing

Inspect signed collision errors and verify inner products are preserved in expectation.

Soft-DTW Alignment Entropy

Soften a time-series alignment into weighted paths and separate expected cost from entropy.

Sparsemax Simplex Projection

Project logits onto a simplex and see exact sparse probabilities emerge.

Spectral Clustering with a Fiedler Cut

Turn a weak graph bridge into a one-dimensional embedding and a two-way cut.

Target Encoding and Leakage

Track exactly which targets are allowed into a categorical encoding.

Tweedie Gaussian Denoising

Turn a marginal density score into a posterior denoising correction.

Bilevel Ridge Hypergradient

Differentiate through an exact trained solution to tune a validation objective.

Bregman Centroid Asymmetry

Find different centroids by reversing the arguments of a convex divergence.

Entropic Mirror Simplex Update

Update probability weights multiplicatively while paying an explicit KL movement cost.

Entropic Transport Coupling Bias

Trade transport cost against coupling entropy and inspect regularization bias.

Hessian–Vector Directional Propagation

Propagate a direction through a gradient graph to reveal curvature without a dense Hessian.

Hoeffding Tree Split Confidence

Compare a split-merit gap with a shrinking confidence threshold before growing a stream tree.

PCGrad Conflicting Task Gradients

Remove conflicting gradient components and inspect each task’s local descent direction.

Random Fourier Gaussian Features

Approximate a Gaussian kernel with shared seeded Fourier features and inspect finite-sample error.

Separable Logistic Finite MLE

Recognize when logistic likelihood has no finite optimizer and how noise or ridge changes existence.

Adaptive Decoupled Weight Decay

With an adaptive metric, L2 regularization and multiplying weights by a decay factor become different updates.

Centered Kernel Alignment Invariance

Compare learned representations through centered sample similarities and test their invariances.

Classifier-Free Gaussian Guidance

Check when an extrapolated score is the gradient of a normalizable density.

Cross-Entropy Elite Gaussian Update

Let elite samples reshape a search distribution and reveal how it can collapse too early.

Dogleg Trust-Region Step

Choose a bounded optimization step along the Cauchy-to-Newton dogleg.

Doubly Robust Policy Value

Combine a reward model with importance-weighted residuals to estimate a target policy value.

EXP3 Bandit Importance Weight Feedback

Seeing only the reward you chose creates biased feedback unless rare choices receive the right importance correction.

FedAvg Quadratic Client Drift

Several local learning steps can converge reliably to the wrong global point when client curvatures differ.

FTRL-Proximal Sparse Logistic Updates

Follow adaptive accumulators to see when an online logistic weight becomes exactly sparse.

GAN Optimal Discriminator JS Identity

An ideal discriminator's objective is exactly a shifted Jensen-Shannon divergence, with perfect separation as a boundary case.

Gauss–Newton Residual Curvature

Separate positive Gauss–Newton curvature from the residual term in the true Hessian.

Gaussian Information Bottleneck Transition

A noisy representation starts preserving useful information only after relevance becomes valuable enough to justify the compression cost.

Generalized Advantage Trace

Decompose an advantage estimate into temporal residuals and distinguish termination from truncation.

LoRA Factor Gradients and Initialization

Inspect the gradients of a low-rank adaptation and explain why one factor starts at zero.

Neural Collapse Simplex Geometry

Move class clouds toward a simplex and separate within-class collapse from loss of class separation.

Noise-Contrastive Density Ratio

Recover a density ratio from binary discrimination while exposing sampling priors and support holes.

NOTEARS Smooth Acyclicity

Detect directed cycles with a smooth matrix-exponential constraint and its gradient.

Oja Hebbian Principal Direction Flow

A local learning rule can stabilize Hebbian growth toward a principal direction, yet a finite numerical step need not preserve its unit norm.

Prioritized Replay Importance Weights

Oversample surprising experiences and see exactly what importance weights do—and do not—correct.

Randomized Smoothing Threshold Certificate

Derive an exact smoothing certificate from Gaussian mass around a one-dimensional threshold.

RealNVP Triangular Density Coupling

An invertible triangular warp makes a flexible density while keeping its inverse and volume correction explicit.

Recursive Least Squares Forgetting

Update a weighted least-squares solution online while tracking the decay of its prior.

ReLU Sharpness Rescaling Symmetry

Keep a neural function identical while making its parameter-space minimum look sharper.

SIREN Nested-Sine Coordinate Derivatives

Differentiate a nested sine network and see how frequency affects the signal and its derivatives.

SURE Soft-Threshold Risk

Choose denoising strength with a risk estimate that does not require knowing the true signal.

UCB Exploration Index Causal Trace

An exploration bonus can revisit an option that looked bad early, while the greedy policy keeps trusting its first impressions.