New dataset and benchmarks for lifelong robotic vision tasks.
problem Challenges in applying computer vision to robots, especially lifelong learning.
method Provided a new lifelong robotic vision dataset and benchmarks.
result Demonstrated the complexity and bottlenecks in lifelong object recognition.
ViWi dataset framework tackles wireless communication problems with visual data.
problem Leveraging visual sensory information in wireless communications.
method Developed a parametric, systematic, and scalable data generation framework using 3D modeling and ray-tracing.
result Offers a way to generate training and testing datasets for assessing machine learning solutions.
Top 8 robotic vision systems tackled lifelong object recognition challenges.
problem Lifelong learning in robotic vision for varied, dynamic environments.
method Design of a dataset with diverse conditions and rules for evaluation.
result Robotic vision systems improved over time with dynamic object appearances.
The paper investigates ethical issues in large image datasets, focusing on pornographic content.
problem Ethical issues in large-scale computer vision datasets, particularly concerning pornographic content.
method Cross-sectional model-based quantitative census covering various factors in the ImageNet-ILSVRC-2012 dataset.
result The dataset contains verifiably pornographic images, including non-consensual and voyeuristic content.
Meta Omnium benchmarks few-shot learning across diverse vision tasks.
problem Evaluating generalization of few-shot learning across multiple vision tasks.
method Introduction of Meta Omnium dataset and evaluation of meta-learning algorithms.
result Meta-learning algorithms can generalize across diverse vision tasks.
IrisNet tracks tongue contours in ultrasound videos using deep learning.
problem Automatic and real-time tracking of tongue contours in ultrasound videos.
method Convolutional neural network mimicking peripheral vision.
result Outstanding generalization on various ultrasound tongue datasets.
Detects anomalies in autonomous mobile robots using vision.
problem Anomaly detection for autonomous mobile robots.
method Unsupervised deep learning methods and a novel dataset.
result State-of-the-art approach tested on a new dataset.
Typically a classifier trained on a given dataset (source domain) does not performs well if it is tested on data acquired in a different setting (target domain). This is the problem that domain adaptation (DA) tries to overcome and, while it is a well explored topic in computer vision, it is largely ignored in robotic …
Plex improves model reliability across vision and language tasks.
problem Improving model reliability in diverse decision-making tasks involving uncertainty and adaptation.
method Developed ViT-Plex and T5-Plex pretrained model extensions to evaluate and improve reliability across 40 datasets.
result Plex greatly improves state-of-the-art across reliability tasks, simplifying evaluation and performance.
Mimetic initialization improves Transformer training on small datasets.
problem Difficulty in training Transformers on small datasets.
method Initialize self-attention layers to look like pre-trained models.
result Vanilla Transformers trained with mimetic initialization achieve higher accuracy.
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
GWHD dataset offers 4,700 high-res images of wheat heads.
problem Challenges in wheat head detection from high-resolution imagery.
method Large, diverse dataset with detailed metadata.
result Benchmark for wheat head detection methods.
We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade. The datasets are collected with a simple adversarial filtration technique to create datasets with limited spurious cues. Our datasets' real-world, unmodified examples transfer to various unseen models …
This paper improves ensemble learning for vision tasks by encouraging diversity in predictions.
problem Generating effective ensembles of neural networks for multi-modal data.
method Explicitly optimize a diversity inducing adversarial loss for learning stochastic latent variables.
result Significant improvements in classification accuracy and out-of-distribution detection compared to baselines.
Transfer learning is a cornerstone of computer vision, yet little work has been done to evaluate the relationship between architecture and transfer. An implicit hypothesis in modern computer vision research is that models that perform better on ImageNet necessarily perform better on other vision tasks. However, this hy…
In this technical report I present my method for automatic synthetic dataset generation for object detection and demonstrate it on the video game League of Legends. This report furthermore serves as a handbook on how to automatically generate datasets and as an introduction on the dataset generation part of the LeagueA…
Language grounded image understanding tasks have often been proposed as a method for evaluating progress in artificial intelligence. Ideally, these tasks should test a plethora of capabilities that integrate computer vision, reasoning, and natural language understanding. However, rather than behaving as visual Turing t…
Fisheye cameras are commonly employed for obtaining a large field of view in surveillance, augmented reality and in particular automotive applications. In spite of their prevalence, there are few public datasets for detailed evaluation of computer vision algorithms on fisheye images. We release the first extensive fish…
Paper explores anchoring for vision models, improving generalization and safety.
problem Anchoring can lead to undesirable shortcuts, limiting generalization.
method Introduced a new anchored training protocol with a regularizer to mitigate undesirable shortcuts.
result Significant performance gains in generalization and safety metrics.
FedVision uses federated learning to improve object detection without transmitting data.
problem Challenges in building object detection models on large training datasets due to privacy and cost issues.
method Federated learning (FL) platform for easy integration by non-experts.
result Significant efficiency improvement and cost reduction in smart city applications.
A new network controls task execution in a single vision system.
problem Executing multiple tasks accurately and efficiently in a single network.
method A top-down control network modifies main recognition network activations based on selected task, image content, and spatial location.
result Significantly better results on four datasets compared to state-of-the-art approaches.
System discovers new classes from unlabeled data, improving model performance.
problem Handling datapoints outside initial training distribution.
method Develops new classes through semi-supervised learning, using Dataset Reconstruction Accuracy and class learnability.
result Demonstrates improved model quality through automatic class discovery.
PACE explains ViTs by modeling patch-level concept distributions, surpassing existing methods.
problem Lack of trustworthy post-hoc explanations for Vision Transformers (ViTs)
method Variational Bayesian explanation framework (PACE)
result PACE surpasses state-of-the-art methods in meeting desiderata for ViT explanations.
Nowadays autonomous technologies are a very heavily explored area and particularly computer vision as the main component of vehicle perception. The quality of the whole vision system based on neural networks relies on the dataset it was trained on. It is extremely difficult to find traffic sign datasets from most of th…
Vision Transformers show different internal representations compared to CNNs.
problem Understanding how Vision Transformers solve image classification tasks.
method Comparative analysis of ViT and CNN architectures on image classification benchmarks.
result ViT has more uniform representations across all layers, while CNNs have more varied representations.
With the advent of large labelled datasets and high-capacity models, the performance of machine vision systems has been improving rapidly. However, the technology has still major limitations, starting from the fact that different vision problems are still solved by different models, trained from scratch or fine-tuned o…
CutMix training technique improves spatial locality in Vision Transformers.
problem Improving spatial locality in Vision Transformers trained from scratch.
method Comparison of Baseline and Modern training protocols on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
result CutMix training component significantly reduces Mean Attention Distance (MAD) in early layers of Vision Transformers.
Few-Shot Diffusion Models generate new samples from small image sets.
problem Generating new samples from a few images.
method Conditional Denoising Diffusion Probabilistic Models (DDPM) with patch-based input set information.
result FSDM can generate samples from previously unseen classes conditioned on as few as 5 samples.
The democratization of machine learning (ML) has led to ML-based machine vision systems for autonomous driving, traffic monitoring, and video surveillance. However, true democratization cannot be achieved without greatly simplifying the process of collecting groundtruth for training and testing these systems. This grou…
Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.
problem Estimating multimodal aleatoric uncertainty in regression tasks from images.
method Regression-by-classification paradigm with hinge-Wasserstein loss.
result Hinge-Wasserstein loss improves uncertainty estimation on challenging tasks.
This work uses image generation models to find vision model bugs.
problem Automatically discovering failures in vision models.
method Conditional text-to-image generation and captioning models.
result Demonstrated utility of large-scale generative models to find vision model bugs.
The computational demands of computer vision tasks based on state-of-the-art Convolutional Neural Network (CNN) image classification far exceed the energy budgets of mobile devices. This paper proposes FixyNN, which consists of a fixed-weight feature extractor that generates ubiquitous CNN features, and a conventional …
New tool uses computer vision to assess FOMC press conference complexity and its impact on equity returns.
problem Understanding the impact of FOMC press conferences on financial markets.
method Developed a measure of discussion complexity using video images and deep learning algorithms.
result Complex discussions during press conferences are associated with higher equity returns and lower volatility.
FMT model improves multimodal sequential learning across language, vision, and acoustic data.
problem Modeling spatio-temporal dynamics across multiple modalities.
method Factorized Multimodal Transformer (FMT) that models intramodal and intermodal dynamics in a factorized manner.
result FMT outperforms existing models on 3 datasets and 21 labels, setting new state of the art.
Simple method disentangles content and style from pre-trained vision models.
problem Learning interpretable features in visual representations.
method Probabilistic linear entanglement model and simple disentanglement algorithm.
result Method provably disentangles content and style features.
Deep Neural Networks(DNN) have excessively advanced the field of computer vision by achieving state of the art performance in various vision tasks. These results are not limited to the field of vision but can also be seen in speech recognition and machine translation tasks. Recently, DNNs are found to poorly fail when …
This paper introduces a novel approach to measuring privacy risks in deep computer vision models based on intermediate outputs.
problem The exposure of intermediate results in hidden layers of deep computer vision models poses significant privacy concerns.
method The approach leverages Degrees of Freedom (DoF) to evaluate the amount of information retained in each layer and combines this with the rank of the Jacobian matrix to assess sensitivity to input variations.
result The proposed framework provides deeper insights into privacy risks associated with intermediate representations without requiring adversarial attack simulations.
Proposes deep collective learning to learn inputs and weights together in neural networks.
problem The need to optimize inputs alongside weights in deep neural networks.
method Introduces deep collective learning to learn inputs and weights jointly, using lookup tables.
result Demonstrates advantages and promise of Lookup-VNets and deep collective learning.
In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as o…
Study evaluates conformal prediction methods for safety in vision models under shifts and long-tailed data.
problem Safety guarantees of conformal prediction methods under distribution shifts and long-tailed data.
method Empirical evaluation of post-hoc and training-based conformal prediction methods on large-scale datasets and models.
result Performance of conformal prediction methods degrades significantly under distribution shifts and long-tailed data.
Novel regularization for Vision Transformers improves model generalization and sparsity.
problem Improving generalization and sparsity in Vision Transformers.
method Likelihood-guided variational Ising-based regularization.
result Improved generalization and sparsity in Vision Transformers.
A benchmark for NLP models trained on text datasets.
problem Limited access to high-performance clusters for NAS experiments.
method Created a search space for recurrent neural networks on text datasets and trained 14k architectures.
result Demonstrated the potential of precomputed NAS results for NLP.
This paper analyzes shallow ViTs, providing sample complexity and SGD behavior insights.
problem Theoretical understanding of shallow ViTs, especially their sample complexity and SGD behavior.
method Data model with label-relevant and label-irrelevant tokens, theoretical analysis of shallow ViT training.
result Characterization of sample complexity for zero generalization error in shallow ViTs.
Cloud based medical image analysis has become popular recently due to the high computation complexities of various deep neural network (DNN) based frameworks and the increasingly large volume of medical images that need to be processed. It has been demonstrated that for medical images the transmission from local to clo…
Proposes MLCNN for better multivariate time series forecasting.
problem Challenges in forecasting multivariate time series, especially the limitation of predicting only one future moment.
method MLCNN, a multi-task deep learning framework inspired by Construal Level Theory, fuses future visions of near and distant future predictions.
result Significant improvements in forecasting accuracy (4.59% RMSE reduction, 6.87% MAE reduction) on real-world datasets.
Inspired by recent successes of deep learning in computer vision, we propose a novel framework for encoding time series as different types of images, namely, Gramian Angular Summation/Difference Fields (GASF/GADF) and Markov Transition Fields (MTF). This enables the use of techniques from computer vision for time serie…
Large-scale labeled training datasets have enabled deep neural networks to excel on a wide range of benchmark vision tasks. However, in many applications it is prohibitively expensive or time-consuming to obtain large quantities of labeled data. To cope with limited labeled training data, many have attempted to directl…
Modern computer vision algorithms often rely on very large training datasets. However, it is conceivable that a carefully selected subsample of the dataset is sufficient for training. In this paper, we propose a gradient-based importance measure that we use to empirically analyze relative importance of training images …