Machine Learning

Recent Publications

Video analysis with the aim of discovering social relations between the people in that video is an important and unexplored topic with significant benefit towards a higher level understanding of videos. This article focuses on the inference of two social groups in each video where members of each group share friendly relations with each other and have an adversarial relation with members of the other social group. Using low-level audiovisual features and motion trajectories we compute a measure of expression of social relation in each scene in video. The occurrence of actors in each scene is computed using face recognition with LBP descriptor. The actor-scene forms a 2-model social network, which we use to compute a 1-mode network of actors. The leaders of each group, which are the actors with greater social impact are estimated using Eigencentrality. We demonstrate our approach on several Hollywood films, which span genres of action, adventure, drama, sci-fi, thriller, historic, and fantasy. This approach is successful at using video content analysis to infer the two social groups and typically the principal protagonist and antagonist in the films as well.
In Academic Press Library in Signal Processing, 2017.

Nuclear power plant (NPP) outages are challenging construction projects. Delays in NPP outage processes can cause significant economic losses. This paper identifies the domain requirements, challenges, and potential solutions of achieving the HCA system that effectively supports resilient NPP outage control. This proposed system aims at signicantly improving the performance of hand-off monitoring/control and responding to contingencies during the outage. Firstly, the authors identified information acquisition and modeling challenges of achieving human-center automation for outage control. The rest of the paper then synthesizes potential techniques available in the domains of computer science, cognitive science, system science, and construction engineering that can potentially address these challenges. The authors concluded this literature and technological review with are search road-map for achieving HCA in construction.
In Automation in Construction, Elsevier, 2017.

Images have become the most popular type of multimedia in the Big Data era. Widely used applications like automatic CBIR underscore the importance of image understanding, especially in terms of the semantically meaningful information. Typically, high dimensional image descriptors are embedded to a subspace using a simple linear projection. However, semantic information has a complex distribution in feature space that requires a non-linear projection. We first estimate an intrinsic dimensionality of image data. Next we build a measure of visual information in embedded subspace. We compare several linear and non-linear projection methods. We use multiple image databases towards a comprehensive evaluation. We report results in terms of information content, consequent recognition rates, and computational cost. This paper is relevant for researchers interested in dimensionality reduction for large scale image understanding that is both quick and preserves semantically relevant information.
In IEEE BigMM, 2017.

Indoor positioning system is a rapidly emerging technology. Unlike outdoor positioning, which uses triangulation from satellites in line-of-sight, current indoor positioning methods attempt triangulation using Received Signal Strength Indicator (RSSI) from indoor transmitters, like WiFi and RFID. These methods, however, are not accurate and suffer from issues like multi-path and absorption by walls and other objects. In this paper we propose an alternate and novel approach to indoor positioning, that combines signals from multiple sensors. In particular, we focus on visual and inertial sensors that are ubiquitously found in mobile devices. We utilize a Building Information Model (BIM) of the indoor environment as a guideline for navigable paths. The sensor suite signals are processed to generate a trajectory of device moving through the indoor environment. We compute features on this trajectory in real-time and data mine pre-computed features on BIM’s navigable paths to determine the location of the device in real-time. We demonstrate our approach on BIM in our university campus. The key benefit of our approach is that unlike previous methods that require installation of a wireless sensor network of several transmitters spanning the indoor environment, we only require a floor-plan BIM and cheap ubiquitous sensor suite on board a mobile device for indoor positioning.

Rapidly growing technologies like autonomous navigation require accurate geo-localization in both outdoor and indoor environments. GNSS based outdoor localization has limitation of accuracy, which deteriorates in urban canyons, forested region and is unavailable indoors. Technologies like RFID, UWB, WiFi are used for indoor localization. These suffer limitations of high infrastructure costs, and signal transmission issues like multi-path, and frequent replacement of transciever batteries. We propose an alternative to localize an individual or a vehicle that is moving inside or outside a building. Instead of mobile RF transceivers, we utilize a sensor suite that includes a video camera and an inertial measurement unit. We estimate a motion trajectory of this sensor suite using Visual Odometery. Instead of pre-installed transceivers, we use GIS map for outdoors, or a BIM model for indoors. The transport layer in GIS map or navigable paths in BIM are abstracted as a graph structure. The geo-location of the mobile platform is inferred by first localizing its trajectory. We introduce an adaptive probabilistic inference approach to search for this trajectory in the entire map with no initialization information. Using an effective graph traversal spawn-and-prune strategy, we can localize the mobile platform in real-time. In comparison to other technologies, our approach requires economical sensors and the required map data is typically available in the public domain. Additionally, unlike other technologies which function exclusively indoors or outdoors, our approach functions in both environments. We demonstrate our approach on real world examples of both indoor and outdoor locations.
In ACM ISA, 2016.

The primary method for geo-localization is based on GPS which has issues of localization accuracy, power consumption, and unavailability. This paper proposes a novel approach to geo-localization in a GPS-denied environment for a mobile platform. Our approach has two principal components: public domain transport network data available in GIS databases or OpenStreetMap; and a trajectory of a mobile platform. This trajectory is estimated using visual odometry and 3D view geometry. The transport map information is abstracted as a graph data structure, where various types of roads are modeled as graph edges and typically intersections are modeled as graph nodes. A search for the trajectory in real time in the graph yields the geo-location of the mobile platform. Our approach uses a simple visual sensor and it has a low memory and computational footprint. In this paper, we demonstrate our method for trajectory estimation and provide examples of geo-localization using public-domain map data. With the rapid proliferation of visual sensors as part of automated driving technology and continuous growth in public domain map data, our approach has the potential to completely augment, or even supplant, GPS based navigation since it functions in all environments.
In ISPRS, 2016.

Nuclear power plant (NPP) outages involve maintenance and repair activities of a large number of workers in limited work spaces, while having tight schedules and zero-tolerance for accidents. During an outage, thousands of workers will be working around the NPP. Extremely high outage costs and expensive delays in maintenance projects (around $1.5 million per day) require tight outage schedules (typically 20 days). In such packed workspaces, real-time human behavior monitoring is critical for ensuring safe collaboration among workers, minimal wastes of time and resources due to the lack of situational awareness, and timely project control. Current methods for detailed human behavior monitoring on construction sites rely on manual imagery data collection and analysis, which is tedious and error-prone. This paper presents a framework of automatic imagery data analysis that enables real-time detection and diagnosis of anomalous human behaviors during outages, through the integration of 4D construction simulation and object tracking algorithms.
In Digital Human Modelling, Springer, 2016.


More Publications

. Social Network Inference in Videos. In Academic Press Library in Signal Processing, 2017.


. Computer Vision Augmented Geospatial Localization. In Encyclopedia of GIS, 2017.

Preprint PDF

. Human-centered automation for resilient nuclear power plant outage control. In Automation in Construction, Elsevier, 2017.

Preprint PDF Video

. Indoor Positioning using Visual and Inertial Sensors. In IEEE SENSORS, 2016.

Preprint PDF

. Ubiquitous Real-Time Geo-Spatial Localization. In ACM ISA, 2016.

Preprint PDF Slides

. GPS-Denied Geo-Localization using Visual Odometry. In ISPRS, 2016.

Preprint PDF Slides

. Automatic Imagery Data Analysis for Diagnosing Human Factors in the Outage of a Nuclear Plant. In Digital Human Modelling, Springer, 2016.

Preprint PDF

Talks & Workshops

GreenR : Plant Health Diagnostics using Image Analysis
May 13, 2016 5:00 PM
GPS-denied Geo-Localization using Visual Odometry and GIS Database
Apr 5, 2016 1:00 PM

Recent Posts




Courses at Ohio State University:

  • CE 5461 : Geo-spatial Numerical Analysis

Assisted courses at University of Surrey:

  • EEE 1025 : Electronic Circuits
  • EEE 2033 : Circuits, Control and Communication
  • EEE 1032 : Mathematics 2: Engineering Mathematics
  • EEE 2035 : Engineering Mathematics 3

Assisted courses at Indian Institute of Technology Kanpur:

  • ESC 102 : Introduction to Electronics
  • EE 678 : Neural Systems and Networks