NSERC Canada Wildfire Strategic Network

Wildfire management is the study of managing, predicting, and mitigating risk of forest wildfires.

DOMAINS | forest-management | forest-wildfire | computational-sustainability

Since 2019 the lab has been part of NSERC Canada Wildfire Strategic Network (https://www.canadawildfire.org/), as part of the more general domain of Forest Wildfire and Remote Sensing.

Specific results from across lab are listed here.

News

Dec 2026: End of an era, the NSERC Wildfire STRATEGIC NETWORK (https://www.canadawildfire.org/) coming to an end, final project report site coming soon.
Aug 2026: Various papers accepted in 2026 on our approaches to upscaling lidar scans of trees from sparse data (details to come).
Aug 2024: MASc student Felix Xiang Fang completes his thesis on

Fang, X. (2024). Canada Wildfire Next-Day Spread Prediction Tools Using Deep Learning [Thesis, University of Waterloo]. In UWSpace Thesis Repository. https://hdl.handle.net/10012/20802
May 2024: CSA Lidar Project approved : see CSA Lidar Project for more information
Fall 2022: PhD Student Sriram Ganapathi Subramanian defended his thesis which utilized Forest Wildfire Spread as one of his application domains:

Subramanian, S. G. (2022). Multi-Agent Reinforcement Learning in Large Complex Environments [Thesis, University of Waterloo]. In UWSpace Thesis Repository. https://doi.org/http://hdl.handle.net/10012/18442
Lab News Dec 12, 2022 : Attended Forest Wildfire Conference on this topic.
Lab News March, 2022 : Two papers related to this topic discussed.
Lab News Dec 1, 2022 : In Fall 2022, members of the lab went to a Canadian Wildfire conference. Mark also gave a talk on research related to this topic.
Journal paper in JAIR on a new algorithm using forest wildfire as one of its motivational examples to demonstrate the difficulty of multi-agent planning in real-world domains with many agents acting at once (Ganapathi Subramanian et al., 2022).
ML for Forest Fire Journal Paper: In 2020, in collaboration with some other investigators in this strategic network, we published a review article in the Environmental Reviews journal (Jain et al., 2020) analysing the relevance of various ML algorithms to the domain and exhaustively surveying and analysing the existing research using ML for forest fire management.
NSERC/CANADA WILDFIRE STRATEGIC NETWORK: In 2019, an NSERC Strategic Network grant was confirmed in which we are involved to support computational research in to Forest Fire Management practices. This includes the AI/ML/RL focus of our research along with our colleague in computer science Prof. Kate Larson.

Completed Theses Related to NSERC Canada Wildfire Strategic Network

MASc Thesis

Canada Wildfire Next-Day Spread Prediction Tools Using Deep Learning

Xiang Fang.

UWSpace Thesis Repository, University of Waterloo, Aug, 2024.

Abs URL

Wildfires have become a pressing issue globally, with their increasing frequency and intensity causing significant environmental, economic, and human impacts. Traditional wildfire prediction methods, while useful, often fall short in time complexity or simulation on heterogeneous landscapes. This thesis explores the application of deep learning models, especially convolutional networks, to improve the accuracy and reliability of wildfire spread predictions. By leveraging advanced machine learning techniques, this research aims to enhance the current prediction capabilities and provide better tools for Canadian wildfire management and mitigation. Utilizing a comprehensive dataset from various sources, this thesis integrates multiple features such as weather data, vegetation types, and topographical information. The research introduces a novel module for fusing multi-modal data, which enhances the performance of U-shape deep learning models like U-Net. Additionally, an innovative U-shape network structure with atrous(dilated) convolution and new attention implementation was developed to further improve prediction accuracy. The thesis also proposes an enhancement method that amplifies grouped error pixels for element-wise error computation for model training. The novel data fusion module proposed in this thesis has been proven to improve the baseline model on the F1 score, while the new model I suggest outperformed the baseline model and its two variants on the same metric. In the final part of the thesis I proposed various additional enhancement methods to improve performance further, it has shown its statistical significance under certain conditions when applied to BCELoss. By enhancing the predictive capabilities of wildfire spread models, this thesis offers valuable insights for emergency responders and policymakers, aiding in better resource allocation and risk mitigation strategies. The deep learning methodologies developed in this study are versatile and have potential applications in other fields requiring spatial data predictions, such as intelligent healthcare, flood forecasting, and disease spread modelling.
PhD Thesis

Multi-Agent Reinforcement Learning in Large Complex Environments

Sriram Ganapathi Subramanian.

UWSpace Thesis Repository, University of Waterloo, Jun, 2022.

Abs URL

Multi-agent reinforcement learning (MARL) has seen much success in the past decade. However, these methods are yet to find wide application in large-scale real world problems due to two important reasons. First, MARL algorithms have poor sample efficiency, where many data samples need to be obtained through interactions with the environment to learn meaningful policies, even in small environments. Second, MARL algorithms are not scalable to environments with many agents since, typically, these algorithms are exponential in the number of agents in the environment. This dissertation aims to address both of these challenges with the goal of making MARL applicable to a variety of real world environments. Towards improving sample efficiency, an important observation is that many real world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. A useful possibility that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this dissertation, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings. To this end, we propose a general model for learning from external advisors in MARL and show that desirable theoretical properties such as convergence to a unique solution concept, and reasonable finite sample complexity bounds exist, under a set of common assumptions. Furthermore, extensive experiments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors. Towards scaling MARL, we explore the use of mean field theory. Mean field theory provides an effective way of scaling multi-agent reinforcement learning algorithms to environments with many agents, where other agents can be abstracted by a virtual mean agent. Prior work has used mean field theory in MARL, however, they suffer from several stringent assumptions such as requiring fully homogeneous agents, full observability of the environment, and centralized learning settings, that prevent their wide application in practical environments. In this dissertation, we extend mean field methods to environments having heterogeneous agents, and partially observable settings. Further, we extend mean field methods to include decentralized approaches. We provide novel mean field based MARL algorithms that outperform previous methods on a set of large games with many agents. Theoretically, we provide bounds on the information loss experienced as a result of using the mean field and further provide fixed point guarantees for Q-learning-based algorithms in each of these environments. Subsequently, we combine our work in mean field learning and learning from advisors to show that we can achieve powerful MARL algorithms that are more suitable for real world environments as compared to prior approaches. This method uses the recently introduced attention mechanism to perform per-agent modelling of others in the locality, in addition to using the mean field for global responses. Notably, in this dissertation, we show applications in several real world multi-agent environments such as the Ising model, the ride-pool matching problem, and the massively multi-player online (MMO) game setting (which is currently a multi-billion dollar market).

Our Papers on NSERC Canada Wildfire Strategic Network

Multi-Advisor-QL

Multi-Agent Advisor Q-Learning

Sriram Ganapathi Subramanian, Kate Larson, Matthew Taylor, and Mark Crowley

In International Joint Conference on Artificial Intelligence (IJCAI) : Journal Track. Macao, China. Aug, 2023.

Abs PDF Code URL Video

In the last decade, there have been significant advances in multi-agent reinforcement learn- ing (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possi- ble. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub- optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experi- ments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors.
Multi-Advisor-MARL

Learning from Multiple Independent Advisors in Multi-agent Reinforcement Learning

Sriram Ganapathi Subramanian, Matthew Taylor, Kate Larson, and Mark Crowley

In Proceedings of the 22nd International Conference on Autonomous Agents and MultiAgent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), London, United Kingdom. Sep, 2023.

Abs PDF URL

Multi-agent reinforcement learning typically suffers from the problem of sample inefficiency, where learning suitable policies involves the use of many data samples. Learning from external demonstrators is a possible solution that mitigates this problem. However, most prior approaches in this area assume the presence of a single demonstrator. Leveraging multiple knowledge sources (i.e., advisors) with expertise in distinct aspects of the environment could substantially speed up learning in complex environments. This paper considers the problem of simultaneously learning from multiple independent advisors in multi-agent reinforcement learning. The approach leverages a two-level Q-learning architecture, and extends this framework from single-agent to multi-agent settings. We provide principled algorithms that incorporate a set of advisors by both evaluating the advisors at each state and subsequently using the advisors to guide action selection. Also, we provide theoretical convergence and sample complexity guarantees. Experimentally, we validate our approach in three different test-beds and show that our algorithms give better performances than baselines, can effectively integrate the combined expertise of different advisors, and learn to ignore bad advice.
Multi-Advisor-QL

Multi-Agent Advisor Q-Learning

Sriram Ganapathi Subramanian, Kate Larson, Matthew Taylor, and Mark Crowley

Journal of Artificial Intelligence Research (JAIR). 74, May, 2022.

Abs PDF Code URL

In the last decade, there have been significant advances in multi-agent reinforcement learning (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possible. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experi- ments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors.
PO-MFRL

Partially Observable Mean Field Reinforcement Learning

Sriram Ganapathi Subramanian, Matthew Taylor, Mark Crowley, and Pascal Poupart.

In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems, London, United Kingdom. May, 2021.

Abs arXiv PDF

Traditional multi-agent reinforcement learning algorithms are not scalable to environments with more than a few agents, since these algorithms are exponential in the number of agents. Recent research has introduced successful methods to scale multi-agent reinforcement learning algorithms to many agent scenarios using mean field theory. Previous work in this field assumes that an agent has access to exact cumulative metrics regarding the mean field behaviour of the system, which it can then use to take its actions. In this paper, we relax this assumption and maintain a distribution to model the uncertainty regarding the mean field of the system. We consider two different settings for this problem. In the first setting, only agents in a fixed neighbourhood are visible, while in the second setting, the visibility of agents is determined at random based on distances. For each of these settings, we introduce a Q-learning based algorithm that can learn effectively. We prove that this Q-learning estimate stays very close to the Nash Q-value (under a common set of assumptions) for the first setting. We also empirically show our algorithms outperform multiple baselines in three different games in the MAgents framework, which supports large environments with many agents learning simultaneously to achieve possibly distinct goals.
WildfireMLRev

A review of machine learning applications in wildfire science and management

Piyush Jain, Sean CP Coogan, Sriram Ganapathi Subramanian, Mark Crowley, Steve Taylor, and Mike D Flannigan.

Environmental Reviews. 28, (3). Canadian Science Publishing, Jul, 2020.

Abs arXiv PDF Slides URL

Artificial intelligence has been applied in wildfire science and management since the 1990s, with early applications including neural networks and expert systems. Since then the field has rapidly progressed congruently with the wide adoption of machine learning (ML) methods in the environmental sciences. Here, we present a scoping review of ML applications in wildfire science and management. Our overall objective is to improve awareness of ML methods among wildfire researchers and managers, as well as illustrate the diverse and challenging range of problems in wildfire science available to ML data scientists. To that end, we first present an overview of popular ML approaches used in wildfire science to date, and then review the use of ML in wildfire science as broadly categorized into six problem domains, including: 1) fuels characterization, fire detection, and mapping; 2) fire weather and climate change; 3) fire occurrence, susceptibility, and risk; 4) fire behavior prediction; 5) fire effects; and 6) fire management. Furthermore, we discuss the advantages and limitations of various ML approaches relating to data size, computational requirements, generalizability, and interpretability, as well as identify opportunities for future advances in the science and management of wildfires within a data science context. In total, we identified 298 relevant publications, where the most frequently used ML methods across problem domains included random forests, MaxEnt, artificial neural networks, decision trees, support vector machines, and genetic algorithms. As such, there exists opportunities to apply more current ML methods — including deep learning and agent based learning — in the wildfire sciences, especially in instances involving very large multivariate datasets. We must recognize, however, that despite the ability of ML models to learn on their own, expertise in wildfire science is necessary to ensure realistic modelling of fire processes across multiple scales, while the complexity of some ML methods, such as deep learning, requires a dedicated and sophisticated knowledge of their application. Finally, we stress that the wildfire research and management communities play an active role in providing relevant, high quality, and freely available wildfire data for use by practitioners of ML methods.