Remote Sensing

Research utilizing image, lidar, radar or other remote sensing data from satellites, planes, or drones.

DOMAINS | forest-management | forest-wildfire | computational-sustainability | autonomous-driving

The broad domain of Sustainable Forest Management, includes a wide variety of tasks such as assessment, modelling, mitigation, prediction, and planning. Prof. Crowley’s own PhD research focussed on this domain from a harvesting pointing of view utilizing probabilistic modelling, simulation, and reinforcement learning. It has remained a core domain of application for his research ever since.

Research on Forestry over the years has fallen into the following groupings, each with their own projects and research:

Forest Wildfire Management - wildfire presents a number of unique challenges which push the boundaries of what is possible with existing AI/ML algorithms. This includes review papers on the use of ML for wildfire, as well as analysis of different fire spread prediction strategies and creation of our own models.
Lidar Forest Scanning - an ongoing project funded by the Canadian Space Agency looking at how to use AI/ML methods to build generative models of forests from multiple types of remote sensing data, including Lidar scans from drones and planes.

News

Completed Theses Related to Remote Sensing

MASc Thesis

Canada Wildfire Next-Day Spread Prediction Tools Using Deep Learning

Xiang Fang.

UWSpace Thesis Repository, University of Waterloo, Aug, 2024.

Abs URL

Wildfires have become a pressing issue globally, with their increasing frequency and intensity causing significant environmental, economic, and human impacts. Traditional wildfire prediction methods, while useful, often fall short in time complexity or simulation on heterogeneous landscapes. This thesis explores the application of deep learning models, especially convolutional networks, to improve the accuracy and reliability of wildfire spread predictions. By leveraging advanced machine learning techniques, this research aims to enhance the current prediction capabilities and provide better tools for Canadian wildfire management and mitigation. Utilizing a comprehensive dataset from various sources, this thesis integrates multiple features such as weather data, vegetation types, and topographical information. The research introduces a novel module for fusing multi-modal data, which enhances the performance of U-shape deep learning models like U-Net. Additionally, an innovative U-shape network structure with atrous(dilated) convolution and new attention implementation was developed to further improve prediction accuracy. The thesis also proposes an enhancement method that amplifies grouped error pixels for element-wise error computation for model training. The novel data fusion module proposed in this thesis has been proven to improve the baseline model on the F1 score, while the new model I suggest outperformed the baseline model and its two variants on the same metric. In the final part of the thesis I proposed various additional enhancement methods to improve performance further, it has shown its statistical significance under certain conditions when applied to BCELoss. By enhancing the predictive capabilities of wildfire spread models, this thesis offers valuable insights for emergency responders and policymakers, aiding in better resource allocation and risk mitigation strategies. The deep learning methodologies developed in this study are versatile and have potential applications in other fields requiring spatial data predictions, such as intelligent healthcare, flood forecasting, and disease spread modelling.
MASc Thesis

Deep Learning and Spatial Statistics for Determining Road Surface Condition

Carrillo Garcia, and Juan Manuel.

UWSpace Thesis Repository, University of Waterloo, Aug, 2019.

Abs URL

Machine Learning (ML), and especially Deep Learning (DL) methods, have evolved rapidly over the last years and showed remarkable advances in research areas such as computer vision and natural language processing; however, there are still engineering applications in industries such as transportation where DL methods have not been applied yet or that can be benefited from an integrated approach using DL in addition to other methods. For countries in Northern latitudes, one of such applications is Monitoring Road Surface Condition (RSC) during the Winter season for improving road safety and road maintenance operations. In this study, we introduce a novel approach for monitoring of RSC that integrates DL methods and Spatial Statistics (SS) to simultaneously process data from roadside cameras and weather stations to determine automatically the category of snow coverage at sample locations across a region of interest. Our approach integrates the advantages of SS for interpolating spatial variables and the strengths of DL for Computer Vision tasks, particularly for image classification. On one hand, SS models serve to understand the spatial autocorrelation of random variables and to determine their expected values in unsampled locations based on a number of near observations. On the other hand, DL models extract relevant patterns from a large number of training images and learn a mapping from input images to a set of predefined labels. We implement and evaluate our approach using data collected in the province of Ontario during the 2017-2018 Winter season. Specifically, we included data from three separate sources, Environment Canada (EC) Weather stations, Road Weather Information System (RWIS) stations, and roadside cameras from the Ministry of Transportation of Ontario (MTO). To the best of our knowledge, this is the first study that integrates both DL and SS techniques for processing the three data sources with the goal of monitoring RSC. The DL models we implement and compared are Inception, Inception-Resnet, Xception, DenseNet, MobileNetv2, and NASNet. All of these models have achieved remarkable results for image classification in well-known benchmarks. The SS models we evaluate are Ordinary Kriging (OK), Radial Basis Functions (RBF), and Inverse Distance Weighted (IDW). The first provides a comprehensive understanding of the spatial autocorrelation for each particular variable, while the second and third allow a faster implementation. Our integrated approach works by combining the output feature vector from the DL model with the interpolated values from the SS model to output a more robust prediction of RSC for the locations of interest.
MASc Thesis

Reinforcement Learning for Determining Spread Dynamics of Spatially Spreading Processes with Emphasis on Forest Fires

Sriram Ganapathi Subramanian.

UWSpace Thesis Repository, University of Waterloo, Apr, 2018.

Abs

Multi-agent reinforcement learning (MARL) has seen much success in the past decade. However, these methods are yet to find wide application in large-scale real world problems due to two important reasons. First, MARL algorithms have poor sample efficiency, where many data samples need to be obtained through interactions with the environment to learn meaningful policies, even in small environments. Second, MARL algorithms are not scalable to environments with many agents since, typically, these algorithms are exponential in the number of agents in the environment. This dissertation aims to address both of these challenges with the goal of making MARL applicable to a variety of real world environments. Towards improving sample efficiency, an important observation is that many real world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. A useful possibility that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this dissertation, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings. To this end, we propose a general model for learning from external advisors in MARL and show that desirable theoretical properties such as convergence to a unique solution concept, and reasonable finite sample complexity bounds exist, under a set of common assumptions. Furthermore, extensive experiments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors. Towards scaling MARL, we explore the use of mean field theory. Mean field theory provides an effective way of scaling multi-agent reinforcement learning algorithms to environments with many agents, where other agents can be abstracted by a virtual mean agent. Prior work has used mean field theory in MARL, however, they suffer from several stringent assumptions such as requiring fully homogeneous agents, full observability of the environment, and centralized learning settings, that prevent their wide application in practical environments. In this dissertation, we extend mean field methods to environments having heterogeneous agents, and partially observable settings. Further, we extend mean field methods to include decentralized approaches. We provide novel mean field based MARL algorithms that outperform previous methods on a set of large games with many agents. Theoretically, we provide bounds on the information loss experienced as a result of using the mean field and further provide fixed point guarantees for Q-learning-based algorithms in each of these environments. Subsequently, we combine our work in mean field learning and learning from advisors to show that we can achieve powerful MARL algorithms that are more suitable for real world environments as compared to prior approaches. This method uses the recently introduced attention mechanism to perform per-agent modelling of others in the locality, in addition to using the mean field for global responses. Notably, in this dissertation, we show applications in several real world multi-agent environments such as the Ising model, the ride-pool matching problem, and the massively multi-player online (MMO) game setting (which is currently a multi-billion dollar market).

Our Papers on Remote Sensing

TreeBench

TreeBench: Evaluating Canopy Structure Preservation in Tree Point Cloud Reconstruction

Josh Qixuan Sun, Erfan Hasanpour Zaryabi, Liam Bennett, Jeff Boisvert, and Mark Crowley

In WACV 2027 Workshop on Computer Vision for Earth Observation (CV4EO). Jan, 2027.

Abs

Tree point cloud reconstruction for forest structural analysis should preserve canopy geometry. Point distances measure proximity and coverage, while preservation of crown extent, vertical structure, and spatial occupancy requires property-specific evaluation. We introduce TreeBench, a benchmark for upsampling and completion derived from 14,884 real-tree observations in FOR-species20K. Using 8,192-point observed references, TreeBench constructs controlled sparse and visibility-masked inputs and provides random and genus-held-out splits. Upsampling models are trained at 4x and evaluated at both 4x and 16x; completion uses low-lateral and overhead visibility patterns. A shared evaluator reports point fidelity and preservation of tree and canopy geometry through 32 metrics. We evaluate 16 baselines, including 15 in all six settings. An unchanged-input control shows that PUCRN reduces random-split 4x stratum-cover profile error from 0.0305 to 0.0061 despite higher Chamfer distance. Selecting models by validation stratum-cover error instead of validation Chamfer distance improves test stratum-cover error in all six settings. Median Chamfer distance also increases on genus-held-out tests while several absolute geometry errors decrease on a shorter tree population. These results show that model comparisons depend on the target canopy property and the test population. TreeBench supports property-specific assessment of reconstruction against observed tree structure.
ISPRS

A Generative Upsampling Framework for Reconstructing High-Density Tree Structures from Low-Density Airborne Lidar

Erfan Hasanpour Zaryabi, Liam Bennett, Josh Qixuan Sun, Mark Crowley, Laura Chasmer, Christopher Hopkinson, and Jeff Boisvert.

In Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.. Copernicus Publications, Toronto, Canada. Jul, 2026.

Abs

Airborne Laser Scanning (ALS) enables efficient large-scale mapping of three-dimensional forest structure due to its extensive spatial coverage, but its low point density often limits the accurate representation of forest structural attributes required for applications such as wildfire management and forest health monitoring. This study proposes a deep generative upsampling framework to enhance low- density ALS tree point clouds and generate dense representations for improved structural analysis. To improve tree reconstruction, the framework incorporates a synthetic pretraining strategy that exposes the model to complete tree geometries before fine-tuning on real data. Experimental results demonstrate that the proposed framework generates high-density tree point clouds with strong spatial similarity to ground-truth point clouds. Synthetic pretraining further improves reconstruction performance, achieving lower Chamfer Distance and higher F1-scores across multiple upsampling rates. In addition, the reconstructed point clouds increase estimation accuracy of key structural attributes, including tree height, crown area, and tree volume. Qualitative experiments on real ALS data confirms the ability of the proposed framework to recover realistic tree structures from sparse point clouds. Overall, the proposed method provides a practical approach for bridging the wide spatial coverage of ALS with the detailed structural information typically obtained from high-density lidar data.
A novel soil moisture retrieval method via combining radiative transfer model and machine learning

Yurun Chen, Cheng Tong, Josh Qixuan Sun, Yulin Shangguan, Xiaodong Deng, Mark Crowley, Hongquan Wang, Yang Ye, Haijun Bao, and Ruqi Huang.

Remote Sensing of Environment. 338, Mar, 2026.

Abs PDF URL

Soil moisture (SM) is a key variable in the global water cycle, and passive microwave remote sensing is widely used for large-scale SM estimation. Although radiative transfer models (RTM) and machine learning (ML) approaches have advanced SM retrieval, interpretable methods that provide a direct, inspectable mapping from satellite observations to SM remain scarce. In this study, we present an initial attempt to develop such an interpretable retrieval framework by fusing the physical principles of the RTM with the mathematical formulation capability of the Kolmogorov–Arnold Network (KAN). The framework proceeds in two stages: first, a physically consistent training dataset is generated by the RTM that links SM with brightness temperature (TB), surface temperature (ST), and vegetation optical depth (VOD), thereby preserving key physical dependencies; second, KAN is trained on this dataset to extract explicit mathematical expressions that relate TB, ST and VOD to SM for different land-cover types. These derived expressions are applied to SMAP observations to produce global daily SM estimates for 2015–2023. Validation against in situ soil moisture measurements from global networks, including the International Soil Moisture Network (ISMN) and the Qinghai Lake Basin dense network (QLB-NET), shows that the KAN retrieval achieves an average correlation coefficient R = 0.64 and unbiased RMSE (ubRMSE) = 0.07 m3/m3, comparable to the SMAP Level-3 product (R = 0.65, ubRMSE =0.06 m3/m3) in capturing broad spatial patterns and seasonal dynamics. Beyond enabling accurate large-scale estimates, the explicit formula extracted by KAN clarifies how TB, ST, and VOD influence SM and provides a compact, direct retrieval expression that requires neither KAN retraining nor the iterative solving typical of traditional RTM approaches, thereby improving reproducibility and operational efficiency. This interpretable fusion of physical modeling and data-driven representation offers a novel pathway for advancing quantitative remote sensing retrievals and invites further refinement and operational evaluation.