Archives for 2020

Today I read a paper titled “On the expressive power of deep neural networks”

The abstract is:
We study the expressivity of deep neural networks with random weights.

We provide several results, both theoretical and experimental, precisely characterizing their functional properties in terms of the depth and width of the network.

In doing so, we illustrate inherent connections between the length of a latent trajectory, local neuron transitions, and network activation patterns.

The latter, a notion defined in this paper, is further studied using properties of hyperplane arrangements, which also help precisely characterize the action of the neural network on the input space.

We further show dualities between changes to the latent state and changes to the network weights, and between the number of achievable activation patterns and the number of achievable labelings over input data.

We see that the depth of the network affects all of these quantities exponentially, while the width appears at most as a base.

These results also suggest that the remaining depth of a neural network is an important determinant of expressivity, supported by experiments on MNIST and CIFAR-10.

Today I read a paper titled “Philosophy in the Face of Artificial Intelligence”

The abstract is:
In this article, I discuss how the AI community views concerns about the emergence of superintelligent AI and related philosophical issues.

Today I read a paper titled “Bandit-Based Random Mutation Hill-Climbing”

The abstract is:
The Random Mutation Hill-Climbing algorithm is a direct search technique mostly used in discrete domains.

It repeats the process of randomly selecting a neighbour of a best-so-far solution and accepts the neighbour if it is better than or equal to it.

In this work, we propose to use a novel method to select the neighbour solution using a set of independent multi- armed bandit-style selection units which results in a bandit-based Random Mutation Hill-Climbing algorithm.

The new algorithm significantly outperforms Random Mutation Hill-Climbing in both OneMax (in noise-free and noisy cases) and Royal Road problems (in the noise-free case).

The algorithm shows particular promise for discrete optimisation problems where each fitness evaluation is expensive.

Today I read a paper titled “Characterization of a Multi-User Indoor Positioning System Based on Low Cost Depth Vision (Kinect) for Monitoring Human Activity in a Smart Home”

The abstract is:
An increasing number of systems use indoor positioning for many scenarios such as asset tracking, health care, games, manufacturing, logistics, shopping, and security.

Many technologies are available and the use of depth cameras is becoming more and more attractive as this kind of device becomes affordable and easy to handle.

This paper contributes to the effort of creating an indoor positioning system based on low cost depth cameras (Kinect).

A method is proposed to optimize the calibration of the depth cameras, to describe the multi-camera data fusion and to specify a global positioning projection to maintain the compatibility with outdoor positioning systems.

The monitoring of the people trajectories at home is intended for the early detection of a shift in daily activities which highlights disabilities and loss of autonomy.

This system is meant to improve homecare health management at home for a better end of life at a sustainable cost for the community.

Today I read a paper titled “A Diagram Is Worth A Dozen Images”

The abstract is:
Diagrams are common tools for representing complex concepts, relationships and events, often when it would be difficult to portray the same information with natural images.

Understanding natural images has been extensively studied in computer vision, while diagram understanding has received little attention.

In this paper, we study the problem of diagram interpretation and reasoning, the challenging task of identifying the structure of a diagram and the semantics of its constituents and their relationships.

We introduce Diagram Parse Graphs (DPG) as our representation to model the structure of diagrams.

We define syntactic parsing of diagrams as learning to infer DPGs for diagrams and study semantic interpretation and reasoning of diagrams in the context of diagram question answering.

We devise an LSTM-based method for syntactic parsing of diagrams and introduce a DPG-based attention model for diagram question answering.

We compile a new dataset of diagrams with exhaustive annotations of constituents and relationships for over 5,000 diagrams and 15,000 questions and answers.

Our results show the significance of our models for syntactic parsing and question answering in diagrams using DPGs.

Today I read a paper titled “Enhanced Twitter Sentiment Classification Using Contextual Information”

The abstract is:
The rise in popularity and ubiquity of Twitter has made sentiment analysis of tweets an important and well-covered area of research.

However, the 140 character limit imposed on tweets makes it hard to use standard linguistic methods for sentiment classification.

On the other hand, what tweets lack in structure they make up with sheer volume and rich metadata.

This metadata includes geolocation, temporal and author information.

We hypothesize that sentiment is dependent on all these contextual factors.

Different locations, times and authors have different emotional valences.

In this paper, we explored this hypothesis by utilizing distant supervision to collect millions of labelled tweets from different locations, times and authors.

We used this data to analyse the variation of tweet sentiments across different authors, times and locations.

Once we explored and understood the relationship between these variables and sentiment, we used a Bayesian approach to combine these variables with more standard linguistic features such as n-grams to create a Twitter sentiment classifier.

This combined classifier outperforms the purely linguistic classifier, showing that integrating the rich contextual information available on Twitter into sentiment classification is a promising direction of research.

Today I read a paper titled “Gearbox Fault Detection through PSO Exact Wavelet Analysis and SVM Classifier”

The abstract is:
Time-frequency methods for vibration-based gearbox faults detection have been considered the most efficient method.

Among these methods, continuous wavelet transform (CWT) as one of the best time-frequency method has been used for both stationary and transitory signals.

Some deficiencies of CWT are problem of overlapping and distortion ofsignals.

In this condition, a large amount of redundant information exists so that it may cause false alarm or misinterpretation of the operator.

In this paper a modified method called Exact Wavelet Analysis is used to minimize the effects of overlapping and distortion in case of gearbox faults.

To implement exact wavelet analysis, Particle Swarm Optimization (PSO) algorithm has been used for this purpose.

This method have been implemented for the acceleration signals from 2D acceleration sensor acquired by Advantech PCI-1710 card from a gearbox test setup in Amirkabir University of Technology.

Gearbox has been considered in both healthy and chipped tooth gears conditions.

Kernelized Support Vector Machine (SVM) with radial basis functions has used the extracted features from exact wavelet analysis for classification.

The efficiency of this classifier is then evaluated with the other signals acquired from the setup test.

The results show that in comparison of CWT, PSO Exact Wavelet Transform has better ability in feature extraction in price of more computational effort.

In addition, PSO exact wavelet has better speed comparing to Genetic Algorithm (GA) exact wavelet in condition of equal population because of factoring mutation and crossover in PSO algorithm.

SVM classifier with the extracted features in gearbox shows very good results and its ability has been proved.

Today I read a paper titled “Font Identification in Historical Documents Using Active Learning”

The abstract is:
Identifying the type of font (e.g., Roman, Blackletter) used in historical documents can help optical character recognition (OCR) systems produce more accurate text transcriptions.

Towards this end, we present an active-learning strategy that can significantly reduce the number of labeled samples needed to train a font classifier.

Our approach extracts image-based features that exploit geometric differences between fonts at the word level, and combines them into a bag-of-word representation for each page in a document.

We evaluate six sampling strategies based on uncertainty, dissimilarity and diversity criteria, and test them on a database containing over 3,000 historical documents with Blackletter, Roman and Mixed fonts.

Our results show that a combination of uncertainty and diversity achieves the highest predictive accuracy (89% of test cases correctly classified) while requiring only a small fraction of the data (17%) to be labeled.

We discuss the implications of this result for mass digitization projects of historical documents.

Today I read a paper titled “Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection”

The abstract is:
We present a novel algorithm for anomaly detection on very large datasets and data streams.

The method, named EXPected Similarity Estimation (EXPoSE), is kernel-based and able to efficiently compute the similarity between new data points and the distribution of regular data.

The estimator is formulated as an inner product with a reproducing kernel Hilbert space embedding and makes no assumption about the type or shape of the underlying data distribution.

We show that offline (batch) learning with EXPoSE can be done in linear time and online (incremental) learning takes constant time per instance and model update.

Furthermore, EXPoSE can make predictions in constant time, while it requires only constant memory.

In addition, we propose different methodologies for concept drift adaptation on evolving data streams.

On several real datasets we demonstrate that our approach can compete with state of the art algorithms for anomaly detection while being an order of magnitude faster than most other approaches.

Today I read a paper titled “Model-driven Simulations for Deep Convolutional Neural Networks”

The abstract is:
The use of simulated virtual environments to train deep convolutional neural networks (CNN) is a currently active practice to reduce the (real)data-hungriness of the deep CNN models, especially in application domains in which large scale real data and/or groundtruth acquisition is difficult or laborious.

Recent approaches have attempted to harness the capabilities of existing video games, animated movies to provide training data with high precision groundtruth.

However, a stumbling block is in how one can certify generalization of the learned models and their usefulness in real world data sets.

This opens up fundamental questions such as: What is the role of photorealism of graphics simulations in training CNN models? Are the trained models valid in reality? What are possible ways to reduce the performance bias? In this work, we begin to address theses issues systematically in the context of urban semantic understanding with CNNs.

Towards this end, we (a) propose a simple probabilistic urban scene model, (b) develop a parametric rendering tool to synthesize the data with groundtruth, followed by (c) a systematic exploration of the impact of level-of-realism on the generality of the trained CNN model to real world; and domain adaptation concepts to minimize the performance bias.

Today I read a paper titled “The Singularity May Never Be Near”

The abstract is:
There is both much optimism and pessimism around artificial intelligence (AI) today.

The optimists are investing millions of dollars, and even in some cases billions of dollars into AI.

The pessimists, on the other hand, predict that AI will end many things: jobs, warfare, and even the human race.

Both the optimists and the pessimists often appeal to the idea of a technological singularity, a point in time where machine intelligence starts to run away, and a new, more intelligent species starts to inhabit the earth.

If the optimists are right, this will be a moment that fundamentally changes our economy and our society.

If the pessimists are right, this will be a moment that also fundamentally changes our economy and our society.

It is therefore very worthwhile spending some time deciding if either of them might be right.

Today I read a paper titled “Optically lightweight tracking of objects around a corner”

The abstract is:
The observation of objects located in inaccessible regions is a recurring challenge in a wide variety of important applications.

Recent work has shown that indirect diffuse light reflections can be used to reconstruct objects and two-dimensional (2D) patterns around a corner.

However, these prior methods always require some specialized setup involving either ultrafast detectors or narrowband light sources.

Here we show that occluded objects can be tracked in real time using a standard 2D camera and a laser pointer.

Unlike previous methods based on the backprojection approach, we formulate the problem in an analysis-by-synthesis sense.

By repeatedly simulating light transport through the scene, we determine the set of object parameters that most closely fits the measured intensity distribution.

We experimentally demonstrate that this approach is capable of following the translation of unknown objects, and translation and orientation of a known object, in real time.

Today I read a paper titled “Sensor Fusion of Camera, GPS and IMU using Fuzzy Adaptive Multiple Motion Models”

The abstract is:
A tracking system that will be used for Augmented Reality (AR) applications has two main requirements: accuracy and frame rate.

The first requirement is related to the performance of the pose estimation algorithm and how accurately the tracking system can find the position and orientation of the user in the environment.

Accuracy problems of current tracking devices, considering that they are low-cost devices, cause static errors during this motion estimation process.

The second requirement is related to dynamic errors (the end-to-end system delay; occurring because of the delay in estimating the motion of the user and displaying images based on this estimate.

This paper investigates combining the vision-based estimates with measurements from other sensors, GPS and IMU, in order to improve the tracking accuracy in outdoor environments.

The idea of using Fuzzy Adaptive Multiple Models (FAMM) was investigated using a novel fuzzy rule-based approach to decide on the model that results in improved accuracy and faster convergence for the fusion filter.

Results show that the developed tracking system is more accurate than a conventional GPS-IMU fusion approach due to additional estimates from a camera and fuzzy motion models.

The paper also presents an application in cultural heritage context.

Today I read a paper titled “Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection”

The abstract is:
We describe a learning-based approach to hand-eye coordination for robotic grasping from monocular images.

To learn hand-eye coordination for grasping, we trained a large convolutional neural network to predict the probability that task-space motion of the gripper will result in successful grasps, using only monocular camera images and independently of camera calibration or the current robot pose.

This requires the network to observe the spatial relationship between the gripper and objects in the scene, thus learning hand-eye coordination.

We then use this network to servo the gripper in real time to achieve successful grasps.

To train our network, we collected over 800,000 grasp attempts over the course of two months, using between 6 and 14 robotic manipulators at any given time, with differences in camera placement and hardware.

Our experimental evaluation demonstrates that our method achieves effective real-time control, can successfully grasp novel objects, and corrects mistakes by continuous servoing.

Today I read a paper titled “Greedy Deep Dictionary Learning”

The abstract is:
In this work we propose a new deep learning tool called deep dictionary learning.

Multi-level dictionaries are learnt in a greedy fashion, one layer at a time.

This requires solving a simple (shallow) dictionary learning problem, the solution to this is well known.

We apply the proposed technique on some benchmark deep learning datasets.

We compare our results with other deep learning tools like stacked autoencoder and deep belief network; and state of the art supervised dictionary learning tools like discriminative KSVD and label consistent KSVD.

Our method yields better results than all.

Today I read a paper titled “Decentralized Optimal Control for Connected and Automated Vehicles at an Intersection”

The abstract is:
In earlier work, we addressed the problem of coordinating online an increasing number of connected and automated vehicles (CAVs) crossing two adjacent intersections in an urban area.

The analytical solution, however, did not consider the state and control constraints.

In this paper, we present the complete Hamiltonian analysis including state and control constraints.

In addition, we present conditions that do not allow the rear-end collision avoidance constraint to become active at any time inside the control zone.

The complete analytical solution, when it exists, allows the vehicles to cross the intersection without the use of traffic lights and under the hard constraint of collision avoidance.

The effectiveness of the proposed solution is validated through simulation in a single intersection and it is shown that coordination of CAVs can reduce significantly both fuel consumption and travel time.

Today I read a paper titled “Towards the Holodeck: Fully Immersive Virtual Reality Visualisation of Scientific and Engineering Data”

The abstract is:
In this paper, we describe the development and operating principles of an immersive virtual reality (VR) visualisation environment that is designed around the use of consumer VR headsets in an existing wide area motion capture suite.

We present two case studies in the application areas of visualisation of scientific and engineering data.

Each of these case studies utilise a different render engine, namely a custom engine for one case and a commercial game engine for the other.

The advantages and appropriateness of each approach are discussed along with suggestions for future work.