Nonlinear dimensionality reduction (NDR or NLDR) is a process of mapping higher-dimensional data into a lower-dimensional non-linear manifold within higher-dimensional space so that the data can be more easily visualized and interpreted. In this context, a manifold is a mathematical space that -- when on a small enough scale -- resembles the Euclidean space of a specific dimension. Manifolds are useful in geometry and mathematical physics because they allow more complicated structures to be expressed and understood in terms of the relatively better-understood properties of simpler spaces.
NDR can be useful because variations in high-dimensional data often has much lower-dimensional explanations, and NDR can help researchers to visualize and understand the underlying structure of the data and the process that generated.
There are two general methods of performing NDR:
- Nonlinearize a linear dimensionality reduction method. (e.g. convert Kernel PCA into nonlinear PCA)
- Use a manifold-based method.
Popular manifold-based methods for nonlinear dimensionality reduction include:
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Joshua B. Tenenbaum, Vin de Silva, John C. Langford
Nonlinear Dimensionality Reduction for Discriminative Analytics of Multiple Datasets
Jia Chen, Gang Wang, Georgios B. Giannakis
On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding
Daniel Ting, Michael I. Jordan
Documentaries, videos and podcasts
Lecture 21: Nonlinear Dimensionality Reduction
December 2, 2011
- Independent component analysis (ICA)A feature extraction technique in signal processing which can be considered a generalization of principal component analysis (PCA), used frequently for the problem of blind signal separation.
- Principal component analysis (PCA)Technique used to find the most valuable parts of all of the variables in a dataset and to then transform the original variables into a smaller set of linear combinations.
- Singular value decomposition (SVD)A popular method of matrix decomposition that reduces a complex matrix into its constituent parts so that subsequent matrix calculations are simpler.
- Non-negative matrix factorization (NMF)A matrix factorization method where all of the values in matrices are constrained to be non-negative so that they are easier to inspect. It is useful in data mining because it has the effect of clustering the input data.
- Factor analysisA method for modeling observed variables and their variance/covariance structures in terms of a smaller number of underlying, unobservable factors.
- Show More