Matthew J. Hoffman

Matthew J. Hoffman

Projects/Data


Semi-Supervised Hyperspectral Object Detection Challenge

(2022) Semi-supervised learning has developed into a highly researched problem as it minimizes the labeling costs while still achieving performance levels comparable to a fully labeled dataset. However, most semi-supervised learning algorithms are based on pre-trained models on ImageNet and are thus challenging to port to other image domains, especially those with more than three bands. In this competition, we present the application of a newly acquired dataset collected from a university rooftop with a hyperspectral camera to perform object detection. As part of the Perception Beyond the Visible Spectrum workshop at the 2022 IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) we organized a semi-supervised learning challenge with 10% labeled data with three moving object categories: vehicles, bus and bike. This dataset was collected as part of the Ph.D. work of Aneesh Rangnekar. The dataset, called, RooftopAI, is a hyperspectral vehicle detection dataset collected with a hyperspectral video platform from the roof of the Chester F. Carlson building on the Rochester Institute of Technology campus over several days. A paper describing the sensor can be found here. The bounding boxes were labelled by Aneesh Rangnekar.

A paper about the challenge results can be found here.

Thanks for the Air Force Office of Scientific Research for supporting this work through the Dynamic Data Driven Application Systems program.

AeroRIT

(2020) This project is part of the Ph.D. work of Aneesh Rangnekar. AeroRIT is a semantic segmentation dataset collected with a hyperspectral sensor over the Rochester Institute of Technology campus. The scene was captured from a Headwall Photonics Micro Hy-perspec E-Series CMOS sensor at an altitude of approximately 5,000 feat (0.4m GSD). The semantic map pixels were labelled by Aneesh Rangnekar with ENVI 4.8.2, using individual hyperspectral signatures and the geo-registered RGB images as references.

 

The link below is to a GitHub page that has AeroRIT data and code.

If using results from this study please cite the following paper: Rangnekar, A., Mokashi, N., Ientilucci, E. J., Kanan, C., & Hoffman, M. J. (2020). AeroRIT: A New Scene for Hyperspectral Image Analysis [Conference Name: IEEE Transactions on Geoscience and Remote Sensing]. IEEE Transactions on Geoscience and Remote Sensing, 58(11), 8116–8124. https://doi.org/10.1109/TGRS.2020. 2987199

Thanks for the Air Force Office of Scientific Research for supporting this work through the Dynamic Data Driven Application Systems program.

Simulating Fate and Transport of Plastic Pollution

(2016-Current) Most of my work here has been on freshwater systems, particualrly the Laurentian Great Lakes. Using currents from NOAA's operational forecast models, we simulated the input and transport of plastic pollution through the Great Lakes. This was the first attempt as a budget for the full Great Lakes system and the first estimates of both input and floating mass of plastic.

Our initial work just looked at surface currents, but in the past few years we've been looking at transport in the water column and deposition on the lake bottom and the beach. In 2020 I, along with math modeling Ph.D. student Juliette Daily, published the first full water column and deposition mass estimates in the Great Lakes or any body of water. We are currently working on developing and improving models of biofouling of particles and beaching/deposition. At RIT I am working on a multidisciplinary effort to model and quantify the deposition of plastic on the bottom of Lake Ontario and impacts on benthic communities. This is suppoerted by NOAA/NY SeaGrant in collaboration with Christy Tyler, Nathan Eddingsaas, Andre Hudson, and Steven Day.

Here is video of simulated plastic particles entering and moving through Lake Erie in the simulation of the year 2009:

Notice how the particles are often pushed along the bottom shore of the Lake but the currents, which leads to higher concentrations along the US coast of Lake Erie. Lake Erie has some gyre structure in its average currents and particles can be seen clustering at certain points in the video, but strong wind events disrupt this pattern and prevent the accumulation of plastic in "garbage patches" as in seen in the oceans.

Here is another video showing simulated plastic particles entering and moving through Lake Michigan, also in 2009:

If using results from this study please cite the following paper: M.J. Hoffman and E. Hittinger. 2016. Inventory and transport of plastic debris in the Laurentian Great Lakes. Marine Pollution Bulletin. In Press.

Simulated Hyperspectral Aerial Video Dataset

(2016-Current) DIRSIG simulated aerial video from a moving platform with moving vehicles. The hyperspectral frames can be downloaded along with the ground truth files for the vehicles. More information on the dataset and our work with it can be found here (citation below).

If using this for reserach, please cite the following paper: Uzkent, B., M.J. Hoffman, and A. Vodacek, 2016. Real-time Vehicle Tracking in Aerial Video using Hyperspectral Features, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop: Moving Cameras Meet Video Surveillance, June 2016.

Here is video of the scene:

Thanks for the Air Force Office of Scientific Research for supporting this work through the Dynamic Data Driven Application Systems program under grant number FA9550-15-1-0093.

email