Sparse Vision MoE matches dense networks in image recognition while using less compute.
problem Scaling vision models efficiently in computer vision.
method Vision MoE (V-MoE) - a sparse version of Vision Transformer.
result V-MoE matches state-of-the-art dense networks in image recognition with half the compute.
Robust CLIP improves vision models' resistance to attacks.
problem Vulnerability of vision models to adversarial attacks.
method Unsupervised adversarial fine-tuning of CLIP.
result Robust CLIP prevents stealth attacks on vision models.
Top 8 robotic vision systems tackled lifelong object recognition challenges.
problem Lifelong learning in robotic vision for varied, dynamic environments.
method Design of a dataset with diverse conditions and rules for evaluation.
result Robotic vision systems improved over time with dynamic object appearances.
Computer vision SSL methods show effectiveness on time series data.
problem Evaluate if computer vision SSL frameworks are effective on time series data.
method Evaluated on UCR and UEA archives, proposed a new method improving VICReg.
result Computer vision SSL frameworks can be effective on time series data.
Quantum hybrid vision transformers improve event classification in high energy physics.
problem Excessive computational resources for training and deploying vision transformer models.
method Constructed quantum hybrid vision transformers for high energy physics event classification.
result Quantum hybrid models achieve comparable performance to classical models with fewer parameters.
Mobile V-MoEs scale down ViTs for resource-constrained vision tasks.
problem Scaling down Vision Transformers for resource-constrained applications.
method Sparse Mixture-of-Experts (MoEs) applied to entire images, with a stable training procedure.
result Mobile V-MoEs achieve better performance-efficiency trade-offs than dense ViTs.
The NIPS 2018 Adversarial Vision Challenge is a competition to facilitate measurable progress towards robust machine vision models and more generally applicable adversarial attacks. This document is an updated version of our competition proposal that was accepted in the competition track of 32nd Conference on Neural In…
GSA-Nets apply group equivariance to self-attention for vision tasks.
problem Improving self-attention networks for vision tasks.
method Define group-equivariant positional encodings.
result GSA-Nets outperform non-equivariant self-attention networks on vision benchmarks.
3DB framework tests and debugs computer vision models using photorealistic simulation.
problem Discovering vulnerabilities and understanding model decision-making in computer vision systems.
method Unified framework using photorealistic simulation to test and debug vision models.
result Insights generated by 3DB transfer to the physical world, enabling robustness analysis.
CViT learns complex physical systems using vision transformer techniques.
problem Learning maps between infinite-dimensional function spaces in scientific machine learning.
method Combines vision transformer encoder, grid-based coordinate embedding, and cross-attention mechanism.
result Achieves state-of-the-art performance on multiple benchmarks, often surpassing larger models.
Computer vision models are unstable due to task symmetries and labelling issues.
problem Instability of computer vision models in classification tasks.
method Analysis of symmetries, categorical nature, and labelling issues.
result Instability is a necessary result of current computer vision formulation.
This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in vision and speech applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent vision and speech systems. With avai…
Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans are prone to similar mistakes. Here, we address this question by leveraging recen…
Computer vision is hard because of a large variability in lighting, shape, and texture; in addition the image signal is non-additive due to occlusion. Generative models promised to account for this variability by accurately modelling the image formation process as a function of latent variables with prior beliefs. Baye…
Visual object detection is a computer vision-based artificial intelligence (AI) technique which has many practical applications (e.g., fire hazard monitoring). However, due to privacy concerns and the high cost of transmitting video data, it is highly challenging to build object detection models on centrally stored lar…
ViLT is a faster vision-and-language model without convolution or region supervision.
problem Efficiency and expressive power limitations in current VLP models.
method A convolution-free Vision-and-Language Transformer (ViLT) that processes visual inputs similarly to textual inputs.
result ViLT is up to tens of times faster with competitive or better performance.
Framework simplifies vision-based control and goal discovery.
problem Learning proportional control from visual data.
method Introduces NewtonianVAE for proportional control and goal discovery.
result Dramatic simplification and acceleration of vision-based controllers.
New framework segments 3D scenes using neural algorithms and sub-Riemannian geometry.
problem Effective scene segmentation in 3D vision.
method Neurogeometric sub-Riemannian model, harmonic analysis, neural-based stereo correspondence.
result Sub-Riemannian metric is central to effective scene segmentation.
Real-time uncertainty estimation for computer vision tasks.
problem Real-time inference of uncertainty in deep learning models.
method Uncertainty-Aware Distribution Distillation method for fast inference.
result Significantly reduced inference time with improved uncertainty and predictive performance.
3D good continuation model explains stereo vision using neurogeometry.
problem Understanding how the brain processes 3D visual correspondence.
method Developed a neurogeometric model involving spatial and orientation disparities.
result Provides insight into neural organization and correspondence problem.
Meta Omnium benchmarks few-shot learning across diverse vision tasks.
problem Evaluating generalization of few-shot learning across multiple vision tasks.
method Introduction of Meta Omnium dataset and evaluation of meta-learning algorithms.
result Meta-learning algorithms can generalize across diverse vision tasks.
The recent breakthroughs in computer vision have benefited from the availability of large representative datasets (e.g. ImageNet and COCO) for training. Yet, robotic vision poses unique challenges for applying visual algorithms developed from these standard computer vision datasets due to their implicit assumption over…
Detects anomalies in autonomous mobile robots using vision.
problem Anomaly detection for autonomous mobile robots.
method Unsupervised deep learning methods and a novel dataset.
result State-of-the-art approach tested on a new dataset.
The growing role that artificial intelligence and specifically machine learning is playing in shaping the future of wireless communications has opened up many new and intriguing research directions. This paper motivates the research in the novel direction of \textit{vision-aided wireless communications}, which aims at …
This work characterizes benign overfitting in Vision Transformers.
problem Understanding generalization of Vision Transformers when trained to overfit.
method Gradient descent on a data distribution model, focusing on self-attention layer and softmax.
result Established a condition to distinguish between small and large test errors based on signal-to-noise ratio.
Vision transformers benefit from non-smooth components in adaptation.
problem Understanding the role of non-smoothness in vision transformer adaptation.
method Theoretical analysis and extensive experiments on large-scale vision transformers.
result High plasticity of attention modules and feedforward layers leads to better finetuning performance.
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
ADAPT improves robustness of Vision Transformers without full model fine-tuning.
problem Vulnerability of Vision Transformers to adversarial attacks.
method Parameter-efficient prompt tuning with ADAPT framework for adaptive adversarial training.
result ADAPT achieves robust accuracy of ~40% w.r.t. SOTA methods using only ~1% of the parameters.
Study proposes a statistical test for Vision Transformer's attention mechanisms.
problem ViT's attention mechanisms may focus on irrelevant regions, leading to unreliable evidence.
method Selective inference framework to quantify statistical significance of attentions as p-values.
result Proposed method enables reliable quantification of false positive detection probability of attentions.
An important goal of computer vision is to build systems that learn visual representations over time that can be applied to many tasks. In this paper, we investigate a vision-language embedding as a core representation and show that it leads to better cross-task transfer than standard multi-task learning. In particular…
B-cos transformers explain Vision Transformers' decisions.
problem Lack of holistic explanations for transformer outputs.
method Formulate each component as dynamic linear, allowing a single linear transform for summarization.
result Bcos-ViTs are highly interpretable and competitive on ImageNet.
Vision Transformers show different internal representations compared to CNNs.
problem Understanding how Vision Transformers solve image classification tasks.
method Comparative analysis of ViT and CNN architectures on image classification benchmarks.
result ViT has more uniform representations across all layers, while CNNs have more varied representations.
This work makes vision networks more interpretable by identifying key neurons.
problem Complex deep networks are hard to interpret, hindering safety-critical applications.
method Uses stochastic local competition and multimodal models to dissect and interpret neurons.
result The method generates understandable descriptions for a few active neurons, revealing network decision-making.
Improves sample efficiency and generalization in vision-based RL by enhancing exploration.
problem Low sample efficiency in vision-based RL using images as observations.
method Integrates state representation learning to enhance exploration and sample diversity.
result Significant improvement in sample efficiency across various environments.
With the advent of large labelled datasets and high-capacity models, the performance of machine vision systems has been improving rapidly. However, the technology has still major limitations, starting from the fact that different vision problems are still solved by different models, trained from scratch or fine-tuned o…
In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execu…
Paper explores anchoring for vision models, improving generalization and safety.
problem Anchoring can lead to undesirable shortcuts, limiting generalization.
method Introduced a new anchored training protocol with a regularizer to mitigate undesirable shortcuts.
result Significant performance gains in generalization and safety metrics.
CutMix training technique improves spatial locality in Vision Transformers.
problem Improving spatial locality in Vision Transformers trained from scratch.
method Comparison of Baseline and Modern training protocols on CIFAR-10, CIFAR-100, and Tiny-ImageNet.
result CutMix training component significantly reduces Mean Attention Distance (MAD) in early layers of Vision Transformers.
This work uses image generation models to find vision model bugs.
problem Automatically discovering failures in vision models.
method Conditional text-to-image generation and captioning models.
result Demonstrated utility of large-scale generative models to find vision model bugs.
This paper tackles shape denoising in computer vision and medical imaging.
problem Handling shapes with missing pieces or outliers in shape modeling.
method Introduces six types of noise and an objective measure for shape denoising.
result Evaluates seven shape denoising methods, six of which are based on deep learning.
Computer vision model automates residual plot assessment for diagnosing model assumptions.
problem Automating residual plot assessment for model diagnostics.
method Trains a computer vision model to predict disparity between residual distributions and reference distributions using Kullback-Leibler divergence.
result Computer vision model is less sensitive to non-linearity but more sensitive than human judgment and conventional tests.
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation…
V-HMN integrates memory mechanisms for improved image recognition.
problem Limited interpretability and high data requirements of existing vision backbones.
method Brain-inspired hierarchical memory modules with iterative refinement.
result V-HMN achieves strong performance on image classification benchmarks.
Paper proposes privacy-preserving learning for images, making them imperceptible to humans but recognizable by machines.
problem Conflict between developing AI systems and protecting sensitive training data.
method Encryption strategies (random shuffling and sub-patch mixing) followed by minimal adaptation to vision transformer.
result Achieves comparable accuracy to competitive methods while ensuring human-imperceptibility of encrypted images.
Interactive framework improves understanding of deep neural networks.
problem False sense of comprehension from static explanation methods.
method Interactive framework allowing exhaustive inspection and testing of decisions.
result Interactive approach leads to better understanding of complex decision boundaries.
Proposes deep collective learning to learn inputs and weights together in neural networks.
problem The need to optimize inputs alongside weights in deep neural networks.
method Introduces deep collective learning to learn inputs and weights jointly, using lookup tables.
result Demonstrates advantages and promise of Lookup-VNets and deep collective learning.
Theoretical analysis of vision transformers' performance with MAE and CL objectives.
problem Understanding the distinct representations learned by vision transformers with MAE and CL objectives.
method Modeling visual data distribution and analyzing ViTs training dynamics with gradient descent.
result ViTs trained with MAE objectives learn both global and local features, while CL-trained ViTs favor global features.
We explore the practicability of Nash's Embedding Theorem in vision and imaging sciences. In particular, we investigate the relevance of a result of Burago and Zalgaller regarding the existence of isometric embeddings of polyhedral surfaces in R3 and we show that their proof does not extended directly to hi…