Current Projects

DARPA Gamebreaker

Collaboration with MIT Prof. Jonathan How. Gamebreaker seeks to develop and apply AI to existing open-world video games to quantitatively assess game balance, identify underlying parameters that significantly contribute to balance, and explore new capabilities, tactics, and rule modifications that are most destabilizing to the game. 2020-2021.

Past Projects

Hierarchical representation of remote sensed multimodal imagery

This project aims to leverage the availability of information at various scales to infer the semantic structure of an image, to explore the use of topological features to improve classification/detection results, and to utilize probabilistic modeling of image content to discover meaningful objects in a given scene by multi-sensor data fusion. Funded by NGA, 2013-2018.

Vision-based sensing for robust localization and mapping for autonomous vehicles

Despite impressive advances in commercial drones and autonomous driving technology, many technical challenges to fully autonomous operation (SAE level 5) remain. Difficulties arise due to severe environmental conditions, such as, variations in illumination (e.g., day, night, dusk) and weather (e.g., rain, fog, snow). Our research focuses on developing robust methods for vision-based simultaneous localization and mapping (SLAM) using monocular and stereo cameras (visual odometry). 2016-2017.

Probabilistic graphical models for hyperspectral analysis

Hyperspectral sensors provide a valuable tool for accurate classification of land cover over large areas. However, major challenges remain due to the high dimensionality, high correlation, and nonlinear spectral measurements. This project proposes to investigate novel inference algorithms to combine spectral and spatial information for hyperspectral image analysis. Funded by Amazon’s AWS and RIT’s seed funding, 2014-2015.

Hyperspectral Analysis of Ore-bearing Rocks

Hyperspectral sensors acquire hundreds of narrow contiguous bands of spectral information over wide areas allowing non-destructive analysis of samples. Research was conducted on applying hyperspectral imaging to map mineral distribution on the surface of an open-pit mine for robot navigation. Funded by Rio Tinto and ARC, 2008-2013.

Rock Characterization using Drill Measurements

A variety of mechanical sensors have been installed on an autonomous drill rig to monitor its performance and operation. Research was conducted on analysing measurement-while-drilling data to gain insight on the relationship between the mechanical measurements and the characteristics of the rocks being drilled. The acquired geological maps can then be used for navigation of other autonomous mining vehicles. Funded by Rio Tinto and ARC, 2008-2013.

Classification of hyperspectral data to identify grass species

An artificial neural network method was developed to classify grass species from hyperspectral image data. The hyperspectral dataset was processed using normalization and second derivative in order to reduce the effect of variations in the intensity level of reflectance and to improve the classification accuracy and generalization performance of the ANN-based model. Funded by JSPS, 2007-2008.

Estimating chemical contents in soybean crops using hyperspectral imagery

This study investigated the prediction of chemical contents in green vegetable soybean (edamame) crops. A neural network-based model was implemented to predict the chemical contents of localized crop fields from hyperspectral data. A novel feature extraction method based on particle swarm optimization was proposed for hyperspectral imagery analysis. Funded by Monbukagakusho, 2005-2006.

See-through-blood hyperspectral/multispectral visualization

An image/signal processing algorithm to be used as aiding tool during surgeries was proposed. The images are obtained using a hyperspectral sensor, which provides data from a large number of measured wavelength bands. A neural network approach was implemented to process hyperspectral data from near infrared wavelengths in order to produce a clearer image of blood covered areas. Funded by Monbukagakusho, 2003-2005.

Home robot mapping and navigation

This work was aimed at developing navigation algorithms for the ApriAlpha Home Robot. Specifically, feature-based and occupancy-grid-based map building algorithms were implemented, and a hybrid mapping approach to allow long-term robot navigation was proposed. This project was a joint collaborative research of Toshiba’s Corporate Research and Development Center and the Tokyo Institute of Technology. Funded by MEXT and Toshiba, 2004-2005.

Reinforcement learning for robot navigation

The performance of reinforcement learning algorithms in a real mobile robot application was analysed. Several algorithms, Q-learning, Sarsa, and Peng-Williams, were tested in a simple task learning experiment. Also, a cognitive map-learning algorithm that used a neural network for sensor fusion and interpretation was implemented. The maps of the real environment acquired by the robot were then used as test bed for the reinforcement learning experiments. Funded by FAPESP, 2001-2002.