Mark Crowley

recent highlighted publications

1. ChemGymRL

   ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry

   Chris Beeler, Sriram Ganapathi Subramanian, Kyle Sprague, Nouha Chatti, Colin Bellinger, Mitchell Shahen, Nicholas Paquin, Mark Baula, Amanuel Dawit, Zihan Yang, Xinkai Li, Mark Crowley, and Isaac Tamblyn.

   In ICML 2023 Synergy of Scientific and Machine Learning Modeling (SynS&ML) Workshop. Jul, 2023.

   Abs PDF Code Poster Slides URL

   This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents ‘on-the-fly’ by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, ChemGymRL, based on the standard Open AI Gym template. ChemGymRL supports a series of interconnected virtual chemical benches where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL.

2. Multi-Advisor-QL

   Multi-Agent Advisor Q-Learning

   Sriram Ganapathi Subramanian, Kate Larson, Matthew Taylor, and Mark Crowley

   In International Joint Conference on Artificial Intelligence (IJCAI) : Journal Track. Macao, China. Aug, 2023.

   Abs PDF Code URL Video

   In the last decade, there have been significant advances in multi-agent reinforcement learn- ing (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possi- ble. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub- optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experi- ments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors.

3. GCRL

   Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting

   Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, and Mark Crowley

   In International Conference on Machine Learning (ICML). Honolulu, Hawaii, USA. Jul, 2023.

   Abs arXiv PDF Code Poster Hypoth

   Conventional supervised learning methods typically assume i.i.d samples and are found to be sensitive to out-of-distribution (OOD) data. We propose Generative Causal Representation Learning (GCRL) which leverages causality to facilitate knowledge transfer under distribution shifts. While we evaluate the effectiveness of our proposed method in human trajectory prediction models, GCRL can be applied to other domains as well. First, we propose a novel causal model that explains the generative factors in motion forecasting datasets using features that are common across all environments and with features that are specific to each environment. Selection variables are used to determine which parts of the model can be directly transferred to a new environment without fine-tuning. Second, we propose an end-to-end variational learning paradigm to learn the causal mechanisms that generate observations from features. GCRL is supported by strong theoretical results that imply identifiability of the causal model under certain assumptions. Experimental results on synthetic and real-world motion forecasting datasets show the robustness and effectiveness of our proposed method for knowledge transfer under zero-shot and low-shot settings by substantially outperforming the prior motion forecasting models on out-of-distribution prediction.

4. IOTSMS

   Aggressive Driver Behavior Detection using Parallel Convolutional Neural Networks on Simulated and Real Driving Data

   Zehra Camlica, Jim Quesenberry, Daniel Carballo, and Mark Crowley

   In 9th International Conference on Internet of Things: Systems, Management and Security (IOTSMS). IEEE, Milan, Italy. Nov, 2022.

   Abs PDF Slides URL

   The novel method proposed in this paper is com- promised of application of two Convolutional Neural Networks (CNN) working in parallel to simultaneously classify driver be- haviors while classifying maneuvers by using time series data. We claim that the Parallel Convolutional Neural Network (PCNN) not only speeds-up training time but also increases performance since having information about the maneuver helps to improve behavior classification performance and vice versa. In this study, both simulation and real-world driving datasets are utilized for driver behavior analysis. As simulation data, mobile phone sensor data are simulated as a time series using a combination of a traffic simulator (SUMO) and a car simulation system (Webots). The same type of data is collected with a specially designed vehicle traveled on a defined route around a predefined region. The collected data are then separately utilized as training and testing data for classification of both maneuvers (e.g turns and lane changes) and driver behaviors (e.g aggressive, non-aggressive) applying a novel method using deep learning on time series data. In addition, other methods which are commonly used for time series analysis, Hidden Markov Models(HMMs) and Recurrent Neural Networks (RNN), are applied to the same datasets to compare with PCNN. According to the results, the CNN classifiers perform efficiently for a single task and PCNN outperforms both single task-CNN and RNN with an average accuracy of 86%.

5. Robot Rescue

   Using Affect as a Communication Modality to Improve Human-Robot Communication in Robot-Assisted Search and Rescue Scenarios

   Sami Alperen Akgun, Moojan Ghafurian, Mark Crowley, and Kerstin Dautenhahn.

   IEEE Transactions on Affective Computing. arXiv, Nov, 2022.

   Abs arXiv PDF URL

   Emotions can provide a natural communication modality to complement the existing multi-modal capabilities of social robots, such as text and speech, in many domains. We conducted three online studies with 112, 223, and 151 participants to investigate the benefits of using emotions as a communication modality for Search And Rescue (SAR) robots. In the first experiment, we investigated the feasibility of conveying information related to SAR situations through robots’ emotions, resulting in mappings from SAR situations to emotions. The second study used Affect Control Theory as an alternative method for deriving such mappings. This method is more flexible, e.g. allows for such mappings to be adjusted for different emotion sets and different robots. In the third experiment, we created affective expressions for an appearance-constrained outdoor field research robot using LEDs as an expressive channel. Using these affective expressions in a variety of simulated SAR situations, we evaluated the effect of these expressions on participants’ (adopting the role of rescue workers) situational awareness. Our results and proposed methodologies provide (a) insights on how emotions could help conveying messages in the context of SAR, and (b) evidence on the effectiveness of adding emotions as a communication modality in a (simulated) SAR communication context.