Mark Crowley

Site being updated, see my Research Lab page for current information.

My research seeks dependable and transparent ways to augment human decision making in complex domains in the presence of spatial structure, large scale streaming data, or uncertainty. My focus is on developing new algorithms within the fields of Reinforcement Learning, Deep Learning and Ensemble Methods, and Manifold Learning (Dimensionality Reduction) as well as other Research Topics.
I often work in collaboration with researchers in applied fields such as Computational Sustainability (including Sustainable Forest Management), Autonomous Driving, AI for Physics (including Combustion Modelling and Digital Chemistry) and Medical Imaging.

note for potential graduate students

The global interest in AI/ML/RL is infectious and it is truly an exciting time to be in research in this field. However, interested students should be aware that I rarely accept new graduate students other than students I already know through courses I teach or other interactions in person. Unfortunately, if you email me with your interest in graduate school I probably will not be able to reply as I receive many such emails every day.

writing

Besides my publications, you can follow my Computationally Thinking blog or @compthink on twitter for links and thoughts on Artificial Intellgience, Machine Learning and how technology and science are advancing.

news

Nov 2, 2021	Recently Accepted Papers more: Three great papers accepted to AAAI, NeurIPS Workshop and EMBC in past month!
Nov 1, 2021	Patent arose in collaboration with DENSO Inc. on a remote controlled car demo for CES. more: Patent awarded for Multi-level collaborative vehicle control
Oct 14, 2021	Autoline.tv interview about our Driver Behaviour Learning project with Magna International. more: Autoline.tv interview about our Driver Behaviour Learning project with Magna International.

selected showcase publications

Decentralized Mean Field Games

Ganapathi Subramanian, Sriram, Taylor, Mathew, Crowley, Mark, and Poupart, Pascal

In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-2022) 2022

Abs

Multiagent reinforcement learning algorithms have not been widely adopted in large scale environments with many agents as they often scale poorly with the number of agents. Using mean field theory to aggregate agents has been proposed as a solution to this problem. However, almost all previous methods in this area make a strong assumption of a centralized system where all the agents in the environment learn the same policy and are effectively indistinguishable from each other. In this paper, we relax this assumption about indistinguishable agents and propose a new mean field system known as Decentralized Mean Field Games, where each agent can be quite different from others. All agents learn independent policies in a decentralized fashion, based on their local observations. We define a theoretical solution concept for this system and provide a fixed point guarantee for a Q-learning based algorithm in this system. A practical consequence of our approach is that we can address a ‘chicken-and-egg’ problem in empirical mean field reinforcement learning algorithms. Further, we provide Q-learning and actor-critic algorithms that use the decentralized mean field learning approach and give stronger performances compared to common baselines in this area. In our setting, agents do not need to be clones of each other and learn in a fully decentralized fashion. Hence, for the first time, we show the application of mean field learning methods in fully competitive environments, large-scale continuous action space environments, and other environments with heterogeneous agents. Importantly, we also apply the mean field method in a ride-sharing problem using a real-world dataset. We propose a decentralized solution to this problem, which is more practical than existing centralized training methods.
MARLEmpircal

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Lee, Ken Ming, Ganapathi Subramanian, Sriram, and Crowley, Mark

Frontiers in Artificial Intelligence (Machine Learning and Artificial Intelligence) 2021
MARLEmpircal

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Lee, Ken Ming, Ganapathi Subramanian, Sriram, and Crowley, Mark

In NeurIPS 2021 Deep Reinforcement Learning Workshop 2021

Abs arXiv

Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on four PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. We show that in fully-observable environments, independent algorithms can perform on par with multi-agent algorithms in cooperative and competitive settings. For the mixed environments, we show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies. We also show that adding recurrence improves the learning of independent algorithms in cooperative partially observable environments.
Acceleration of Large Margin Metric Learning for Nearest Neighbor Classification Using Triplet Mining and Stratified Sampling

Poorheravi, Parisa, Ghojogh, Benyamin, Gaudet, Vincent, Karray, Fakhri, and Crowley, Mark

Journal of Computational Vision and Imaging Systems 2021

Abs PDF URL

Metric learning is a technique in manifold learning to find a projection subspace for increasing and decreasing the inter- and intra-class variances, respectively. Some metric learning methods are based on triplet learning with anchor-positive-negative triplets. Large margin metric learning for nearest neighbor classification is one of the fundamental methods to do this. Recently, Siamese networks have been introduced with the triplet loss. Many triplet mining methods have been developed for Siamese nets; however, these techniques have not been applied on the triplets of large margin metric learning. In this work, inspired by the mining methods for Siamese nets, we propose several triplet mining techniques for large margin metric learning. Moreover, a hierarchical approach is proposed, for acceleration and scalability of optimization, where triplets are selected by stratified sampling in hierarchical hyper-spheres. We analyze the proposed methods on three publicly available datasets.
Analysis of Language Embeddings for Classification of Unstructured Pathology Reports

Allada, Aishwarya Krishna, Wang, Yuanxin, Jindal, Veni, Babee, Morteza, Tizhoosh, H.R., and Crowley, Mark

In International Conference of the IEEE Engineering in Medicine and Biology Society 2021

Abs

A pathology report is one of the most significant medical documents providing interpretive insights into the visual appearance of the patient’s biopsy sample. In digital pathology, high-resolution images of tissue samples are stored along with pathology reports. Despite the valuable information that pathology reports hold, they are not used in any systematic manner to promote computational pathology. In this work, we focus on analyzing the reports, which are generally unstructured documents written in English with sophisticated and highly specialized medical terminology. We provide a comparative analysis of various embedding models like BioBERT, Clinical BioBERT, BioMed-RoBERTa and Term Frequency-Inverse Document Frequency (TF-IDF), a traditional NLP technique, as well as the combination of embeddings from pre-trained models with TF-IDF. Our results demonstrate the effectiveness of various word embedding techniques for pathology reports.

Tweets by compthink