Manifold Learning

Manifold learning looks at ways to automatically extract meaningful features, dimensions or subspaces from data in order to build better models, expand data, reduce data, etc.

WEBPAGE: Elements of Dimensionality Reduction and Manifold Learning (Amazon)

Manifold Learning and Dimensionality Reduction are vast areas of study in Math and Computer Science. The task is to find ways to determine the essential relationships and structure of a dataset. Researchers in this area looks at ways to automatically extract meaningful features, dimensions or subspaces from data in order to build better models, expand data, reduce data, etc.

The recent focus on Deep Learning seems to raise the question whether dedicated research on Manifold Learning and Dimensionality Reduction are still required as their own pursuit since. After all, some form of Encoder-Decoder neural network could always be devised as a replacement. While such systems work well given the right training process and enough data, there is also certainly a role to be played by interpretable models built on solid statistical concepts.

Extraction of lower-dimensional representations of data can allow more compact storage or transmission and also improve the performance of other ML tasks such as classification and regres_sion, as the more compact representation must necessarily encode the most important relationships to maintain accuracy.

We have an exciting group of work which has been published in recent years on this topic which you can see below in the Publications list.

Upcoming Textbook on Manifold Learning!

This work has culiminated recently in the graduation of my first Doctoral student, Benyamin Ghojogh, in April 2021 with his thesis encompassing many of these advances. Dr. Ghojogh continued as a postdoc in my lab until 2022 and now works in industry. In late 2022 we will publish, via Springer, a new textbook on “Manifold Learning and Dimensionality Reduction” (Ghojogh et al., 2023) written in collaboration with Prof. Ali Godsi andd Prof. Fakhri Karray.

Our Papers on Manifold Learning

  1. QQE
    Quantile–Quantile Embedding for distribution transformation and manifold embedding with ability to choose the embedding distribution
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    Machine Learning with Applications (MLWA). 6, 2021.
  2. Textbook
    Elements of Dimensionality Reduction and Manifold Learning
    Springer Nature, Feb, 2023.
  3. Theoretical Connection between Locally Linear Embedding, Factor Analysis, and Probabilistic PCA
    In Canadian Conference on Artificial Intelligence. Canadian Conference on Artificial Intelligence (CAIAC), Toronto, Ontario, Canada. May, 2022.
  4. TOOL-Gen-LLE
    Generative locally linear embedding: A module for manifold unfolding and visualization
    Software Impacts. 9, (100105). Elsevier, 2021.
  5. Acceleration of Large Margin Metric Learning for Nearest Neighbor Classification Using Triplet Mining and Stratified Sampling
    Parisa Poorheravi, Benyamin Ghojogh, Vincent Gaudet, Fakhri Karray, and Mark Crowley
    Journal of Computational Vision and Imaging Systems. 6, (1). Jan, 2021.
  6. Batch-Incremental Triplet Sampling for Training Triplet Networks Using Bayesian Updating Theorem
    Milad Sikaroudi, Benyamin Ghojogh, Fakhri Karray, Mark Crowley, and H. R. Tizhoosh.
    In 25th International Conference on Pattern Recognition (ICPR). IEEE, Milan, Italy (virtual). Jan, 2021.
  7. QQE
    Quantile–Quantile Embedding for distribution transformation and manifold embedding with ability to choose the embedding distribution
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    Machine Learning with Applications (MLWA). 6, 2021.
  8. Weighted Fisher Discriminant Analysis in the Input and Feature Spaces
    Benyamin Ghojogh, Milad Sikaroudi, H.R. Tizhoosh, Fakhri Karray, and Mark Crowley
    In International Conference on Image Analysis and Recognition (ICIAR-2020). Springer, Póvoa de Varzim, Portugal (virtual). Jun, 2020.
  9. Theoretical Insights into the Use of Structural Similarity Index In Generative Models and Inferential Autoencoders
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    In International Conference on Image Analysis and Recognition (ICIAR-2020). Springer, Póvoa de Varzim, Portugal (virtual). Jun, 2020.
  10. Generalized Subspace Learning by Roweis Discriminant Analysis
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    In International Conference on Image Analysis and Recognition (ICIAR-2020). Springer, Póvoa de Varzim, Portugal (virtual). Jun, 2020.
  11. Instance Ranking and Numerosity Reduction Using Matrix Decompositionand Subspace Learning
    Benyamin Ghojogh, and Mark Crowley
    In Canadian Conference on Artificial Intelligence. Springer’s Lecture Notes in Artificial Intelligence., Kingston, ON, Canada. 2019.
  12. Locally Linear Image Structural Embedding for Image Structure Manifold Learning
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    In International Conference on Image Analysis and Recognition (ICIAR-19). Waterloo, Canada. 2019.
  13. Image Structure Subspace Learning Using Structural Similarity Index
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    In International Conference on Image Analysis and Recognition (ICIAR-19). Waterloo, Canada. 2019.
  14. Principal Component Analysis Using Structural Similarity Index for Images
    Benyamin Ghojogh, Fakhri Karray, and Mark Crowley
    In International Conference on Image Analysis and Recognition (ICIAR-19). Waterloo, Canada. 2019.
  15. Principal Sample Analysis for Data Reduction
    Benyamin Ghojogh, and Mark Crowley
    In 2018 IEEE International Conference on Big Knowledge (ICBK). Singapore. 2018.