Artificial Intelligence Research

Artificial Intelligence research at UR includes work on knowledge representation and reasoning, machine learning, dialog systems, statistical natural language processing, automated planning, AI-based assistive technology, and computer vision. We collaborate with the departments of linguistics and brain and cognitive sciences.  We participate in the Center for Language Sciences, the Center for Visual Science, the Center for Computation and the Brain, and The Goergen Institute for Data Science.


James Allen

James' research interests span a range of issues covering natural language understanding, discourse, knowledge representation, common-sense reasoning, and planning. He has joint appointments in the brain and cognitive sciences and linguistics departments, holds the Dessauer Chair in Computer Science, and is a fellow of the AAAI. He is the author of the definitive graduate-level textbook Natural Language Understanding, 2nd ed. (Benjamin Cummings, 1994).

Chris Brown

Chris's current activities center around building behaving, visually guided systems. Recent projects include image inpainting, video segmentation, inferring depth from trajectories in multi-view imaging, and digital restoration of historic images.

Daniel Gildea

Dan is interested in statistical approaches to natural language processing, in particular language understanding and machine translation. He has also worked on language and pronunciation modeling for speech recognition and computational approaches to phonology.

Henry Kautz

Henry’s research projects include data mining social media in order to track disease and improve public health; grounded language learning by align text and video; and knowledge representation and reasoning systems that combine logic and probability.  He is the Robin & Tim Wentworth Director of the Goergen Institute for Data Science, and a Fellow of the American Association for the Advance of Science (AAAS).

Ji Liu

Ji Liu's research is focused on:

  • Optimization - asynchronous parallel methods for big data applications, large scale problems, first order algorithms, stochastic algorithms
  • Machine Learning - compressed sensing theorytensor completion, tensor decomposition, multi-tasks learning, reinforcement learning
  • Computer Visionspectral clustering, abnormal event detection, online metric learning, dictionary learning, image segmentation, image inpainting
Jiebo Luo

Jiebo's research spans image processing, computer vision, machine learning, social media, data mining, medical imaging, and ubiquitous computing. He has been an advocate for contextual inference in semantic understanding of visual data, and continues to push the frontiers in this area by incorporating geo-location context and social context. A recent research thrust focuses on exploiting social media for machine learning, data mining, and human-computer interaction, for example, mining the wisdom of crowds for social, political, and economic prediction and forecasting. He has published extensively with over 250 papers and 80 US patents.

Randal Nelson

Randal's research interests involve machine vision and robotics with an emphasis on systems that engage in sensory-mediated interaction with the physical world. Current and recent projects include work on object recognition and learning representations at the interface between feature-based and appearance-based approaches, vision for manipulation and hand-eye coordination, motion recognition and analysis, and visual navigation.

Lenhart Schubert

Len's research interests center around language, knowledge representation, inference and planning. These interests are tied together by the general goal of developing agents with common sense and the ability to converse and acquire knowledge through language.

Chenliang Xu

Chenliang’s research thrusts include computer vision and its relations to natural language, robotics and data science. He primarily focuses on problems in video understanding such as video segmentation, activity recognition, and multimodal vision-and-x modeling. Recent projects include work on cross-modal audio-visual generation, fine-grained actor-action segmentation, and video storytelling.

Project Pages

Project NameBrief Summary
Action Recognition from Unconstrained Amateur Videos

We developed a systematic framework for recognizing realistic actions from unconstrained amateur videos which have tremendous variations due to camera motion, background clutter, changes in object appearance and scale, and so on.

Boundary Extraction by Lineal Feature Growing

Computer vision method for extracting lineal features, both curved and straight, from an image using extended local information to provide robustness and sensitivity.

Digital Analysis and Restoration of Daguerreotypes

Cluster computing allows standard digital analysis and restoration techniques to be applied to high-resolution microscopic digitizations of Daguerreotypes from the collection of the George Eastman House in Rochester. Knowing the image context of a feature (such as a small light spot) affects its probability of being noise (dust effect) or signal (foliage effect). Machine learning can be used to automate some subtle decisions.

Interactive Co-segmentation of Topically Related Images with Intelligent Scribble Guidance

We developed a user-friendly system to facilitate a user to perform interactive segmentation of objects of interest from a group of related images by providing scribble guidance.


Grounded language learning by aligning text and video.

Mining the Power of 'Like' in Social Media Networks

'Like' has now become a very popular social function on social media networks by allowing users to express their positive opinions of certain objects. It provides an accurate way of gauging user interests and an effective way of sharing or promoting information in social media. We developed a system called LikeMiner using a heterogeneous network model and related mining algorithms to estimate the representativeness and influence of objects.

Planning as Satisfiability

Solving combinatorially challenging planning problems by encoding as Boolean satisfiability and applying state of the art SAT solvers.

Machine Translation

Methods for translating between natural languages (such as English and Chinese) by training statistical models on large collections of text.


The TRAINS project and its successors form one of the longest running research efforts on practical spoken dialogue: conversation undertaken with a specific task in mind.

Image Sentiment Analysis using Progressively Trained and Domain Transferred Deep Networks

We developed a robust algorithm for predicting the sentiment carried by an image using a progressive training strategy to derive a convolution neural network from samples with noisy sentiment labels

Leveraging User Generated Online Photos to Estimate and Monitor Air Pollution in Major Cities

We developed a novel system that makes use of everyday user shared photos to measure the air quality of different locations.