Locally Linear Embeding (LLE) Vs Locality Preserving Projections(LPP)

References:

[1] Saul and Roweis: Think Globally, Fit Locally, Unsupervised Learning of Nonlinear Manifolds (U. Penn. Tech Report CIS-02-18).

[2] He, Yan, Hu, Niyogi, and Zhang. Face Recognition Using Laplacianfaces.


Saul and Roweis [1] presents a method to compute a low dimensional embedding of high dimensional data assumed to lie on non linear manifold. It tries to preserve the property that nearby points in the high dimensional space remain nearby and similarly co-located with respect to one another in the low dimensional space.

Laplacian face paper [2] extends the idea of preserving local neighborhood distance by obtaining a subspace of given high dimensional data which may lie of manifold.
More specifically, the manifold structure is modeled by a nearest-neighbor graph which preserves the local structure of the image space. A face subspace is obtained
by Locality Preserving Projections (LPP). Each face image in the image space is mapped to a low dimensional face subspace, which is characterized by a set of feature images, called Laplacian faces. They claim that their method is the first method to face analysis, which explicitly considers the nonlinear manifold structure of the data.

The main contribution of Laplacian face approach is that the method can handle supervised learning tasks like face recognition and even novel data can be represented in the computed subspace. Methods based on LLE yield maps that are defined only on the training data points and its evaluation on novel test data points is not so clear.
Although Saul and Roweis suggests a way to generalize LLE to novel data points using non-parametric model and parametric model, the former has a big disadvantage that it requires access to the entire set of previously analyzed inputs and outputs and hence potentially a large demand in storage. Parametric model using mixture models are also suggested but obtaining a global coordinate system by patching together the local coordinate systems of individual components in mixture model is difficult and unclear.

LLE does not explicitly consider the structure of the manifold on which the data/images possibly reside. Kernel based techniques for face recognition can discover the nonlinear structure of the face images but they are computationally expensive.

Hence in my opinion the Laplacian method has advantages in supervised learning and applications where knowing the structure of manifold is important, while LLE is good for dimensionality reduction and visualization of a given data.