We study how to leverage off-the-shelf visual and linguistic data to cope with out-of-vocabulary answers in visual question answering task. Existing large-scale visual datasets with annotations such as image class labels, bounding boxes and region descriptions are good sources for learning rich and diverse visual conce…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “visual modelling”
VINE visualizes statistical interactions in complex models.
problem Lack of utilities for regional explanations in black box models.
method VINE algorithm to extract and visualize statistical interaction effects.
result VINE provides a novel evaluation metric for visualizations.
Visualizes deep generative models for drug design.
problem Limited visualization tools for deep generative models in drug discovery.
method Proposes a visualization framework for deep graph generative models.
result Interactive visualization and molecular optimization tools.
AV-CPL uses continuous pseudo-labels for AVSR combining labeled and unlabeled data.
problem Improving AVSR performance with labeled and unlabeled data.
method Semi-supervised method using continuous pseudo-labels generated by the same AVSR model.
result Significant improvements in VSR performance on LRS3 dataset.
The abstract discusses how humans use visualizations in machine learning.
problem The reliance on human involvement in AI systems and analytics.
method Review of seven steps in the ML process and different visualization techniques.
result Different visualizations are used at various stages of the ML process.
New report on machine learning visualization techniques and trends.
problem Improving trust in machine learning models through visualization.
method Analysis of peer-reviewed articles on machine learning visualization techniques.
result Rapid growth in machine learning visualization techniques over the past three years.
Pretrained model improves visual dialog performance.
problem Improving performance in visual dialog tasks.
method Pretrained ViLBERT model on vision-language datasets, fine-tuned on VisDial.
result Best model outperforms prior work by more than 1% on NDCG and MRR.
Speaker-Independent Speech-Driven Visual Speech Synthesis using Domain-Adapted Acoustic Modelseess.AS
Improved visual speech synthesis using adapted ASR acoustic models.
problem Lack of synchronized audio, video, and depth data for speaker-independent speech-driven visual speech synthesis.
method Adapted an ASR acoustic model trained on audio-only data to the visual speech synthesis domain.
result Viewers significantly prefer animations generated from the adapted ASR acoustic model.
Method generates visual explanations for similarity models without classification.
problem Lack of visual explanations for similarity models trained without classification loss.
method Gradient-based visual attention using learned feature embeddings.
result Attention maps improve model performance and can be used as constraints.
Tool visualizes Transformer model attention for better understanding.
problem Understanding complex attention mechanisms in deep learning models.
method Developed an open-source tool to visualize attention at three levels.
result Visualization helps interpret and analyze Transformer models.
Speech enhancement (SE) aims to reduce noise in speech signals. Most SE techniques focus only on addressing audio information. In this work, inspired by multimodal learning, which utilizes data from different modalities, and the recent success of convolutional neural networks (CNNs) in SE, we propose an audio-visual de…
Speech enhancement (SE) aims to reduce noise in speech signals. Most SE techniques focus only on addressing audio information. In this work, inspired by multimodal learning, which utilizes data from different modalities, and the recent success of convolutional neural networks (CNNs) in SE, we propose an audio-visual de…
System helps scientists visualize deep learning model of x-ray images.
problem Understanding complex x-ray scattering images with multiple attributes.
method Interactive visualization system in feature space and classification output.
result Users can explore and compare images and attributes flexibly.
IANN visualizes all input variables effects simultaneously.
problem Inability to visualize all input variables effects simultaneously in black-box functions.
method Interpretable Architecture Neural Network (IANN) approach.
result Visualization of all input variables effects directly and simultaneously.
Recent studies in the field of human vision science suggest that the human responses to the stimuli on a visual display are non-deterministic. People may attend to different locations on the same visual input at the same time. Based on this knowledge, we propose a new stochastic model of visual attention by introducing…
A new geometric model for V1 hypercolumns combines symplectic and spherical models.
problem Understanding the structure of V1 hypercolumns in the visual cortex.
method A differential geometric model based on conformal geometry.
result Combines features of symplectic and spherical models of hypercolumns.
New model for visual cortex border completion using bicycle wheel motions.
problem Understanding border completion in the visual cortex V1.
method Sub-Riemannian Hamiltonian formalism and bicycle wheel analogy.
result Analogies between visual cortex border completion and bicycle wheel motions.
New method visualizes decision boundaries of classification models.
problem Difficulty in understanding how classification models interpret data.
method Hybrid supervised-unsupervised technique for visualizing decision boundaries.
result Provides interpretable maps for qualitative and quantitative analysis.
ExMatrix visualizes RF models, aiding in decision understanding.
problem Lack of interpretability in complex models like RF.
method ExMatrix, a novel visualization method for RF interpretability.
result ExMatrix handles large, complex RF models and promotes interpretability.
We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize…
We consider the problem of naming objects in complex, natural scenes containing widely varying object appearance and subtly different names. Informed by cognitive research, we propose an approach based on sharing context based object hypotheses between visual and lexical spaces. To this end, we present the Visual Seman…
This work learns visual representations for deformable objects using contrastive estimation.
problem Challenges in learning plannable visual representations for deformable objects.
method Jointly optimizes visual representation and dynamics models using contrastive estimation.
result Substantial improvements in performance over standard model-based learning techniques.
Neural model predicts object states and physical parameters from visual observations.
problem Computational models struggle with physical reasoning and adapting to new environments.
method Visual prior predicts particle-based system from visual observations; inference module refines estimates subject to dynamics constraints.
result Model can infer physical properties within a few observations and adapt to unseen scenarios.
Paper establishes baselines for offline RL from visual observations.
problem Challenges in offline reinforcement learning from visual observations with continuous action spaces.
method Simple baselines and benchmarking tasks for offline RL from visual observations.
result Simple modifications to existing online RL algorithms outperform existing offline RL methods.
InstanceFlow visualizes classifier confusion over training epochs.
problem Limited model interpretability through aggregate performance measures.
method Dual-view visualization tool showing instance-level learning behavior.
result Allows temporal analysis of training process and instance-level performance.
Methodology for visualizing labeled datasets with mixed features.
problem Visualization of labeled mixed-featured datasets.
method Developed a Max-Ratio Projection (MRP) method for continuous features and extended it to datasets with discrete and continuous features using Gaussianized distributional transforms and copula models.
result Visualization of labeled mixed-featured datasets using Max-Ratio Projection and Gaussianized distributional transforms.
Visualizes deep neural networks for speech recognition using learned topographic filter maps.
problem Unintuitive internal structure of deep neural networks complicates activation visualization.
method Trains a convolutional speech recognition model with filters arranged in a 2D grid, highlighting similar filters.
result Topographic filter maps visualize artificial neuron activations more intuitively.
The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizationscs.LG
Report on enhancing trust in ML models with visualizations.
problem Understanding and trusting the results of black box ML models.
method Categorization of visualization techniques, statistical overview, topic analyses, and interactive web-based survey.
result Expanded categorization of trust against different facets of interactive ML.
Model learns multiple tasks using visual and textual representations.
problem Training visual navigation agents for multiple tasks.
method Dual-Attention unit for task-invariant alignment of visual and textual representations.
result Model outperforms baselines on semantic goal navigation and embodied question answering.
Visual question answering is a recently proposed artificial intelligence task that requires a deep understanding of both images and texts. In deep learning, images are typically modeled through convolutional neural networks, and texts are typically modeled through recurrent neural networks. While the requirement for mo…
A framework isolates VQA reasoning from perception for better model evaluation.
problem Improper separation of visual perception and reasoning in VQA models.
method Introducing a framework and a top-down calibration technique to decouple reasoning from perception.
result Improved evaluation of VQA models by separating reasoning from perception.
The task of associating images and videos with a natural language description has attracted a great amount of attention recently. Rapid progress has been made in terms of both developing novel algorithms and releasing new datasets. Indeed, the state-of-the-art results on some of the standard datasets have been pushed i…
Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networkscs.CV
Interpretation and explanation of deep models is critical towards wide adoption of systems that rely on them. In this paper, we propose a novel scheme for both interpretation as well as explanation in which, given a pretrained model, we automatically identify internal features relevant for the set of classes considered…
The paper uses deep learning to speed up spatial and visual connectivity analysis.
problem Slow calculation of spatial and visual connectivity metrics.
method Investigates machine learning models and a pipeline for training them on spatial and visual connectivity analysis.
result Deep learning models significantly speed up the analysis process.
A faster method for visualization recommendations on large datasets.
problem Infeasibility of state-of-the-art vis-rec models on large datasets due to high computational time.
method Reinforcement-learning (RL) framework that identifies optimal statistics within a time budget.
result Significantly reduces time-to-visualize with minimal error compared to baseline approaches.
We propose a novel methodology, forest floor, to visualize and interpret random forest (RF) models. RF is a popular and useful tool for non-linear multi-variate classification and regression, which yields a good trade-off between robustness (low variance) and adaptiveness (low bias). Direct interpretation of a RF model…
A framework visualizes embedding spaces of neural survival analysis models using anchor directions.
problem Visualizing complex embeddings in neural survival analysis models.
method Estimating anchor directions through clustering or user-supplied concepts, revealing relationships with raw inputs and survival times.
result Visualization strategies reveal how anchor directions relate to raw clinical features and survival time distributions.
regvis.net offers a visual survey of regulatory visualization.
problem Lack of a comprehensive resource for regulatory visualization.
method Collection and manual tagging of 80+ publications, creation of a searchable webpage.
result First publication set tailored for regulatory visualization.
This paper discusses the role of risk communication in macroprudential oversight and of visualization in risk communication. Beyond the soar in data availability and precision, the transition from firm-centric to system-wide supervision imposes vast data needs. Moreover, except for internal communication as in any orga…
Twitmo analyzes geo-tagged Twitter data for topic modeling and visualization.
problem Analyzing public discourse on Twitter for various topics, parties, or individuals.
method Collects and preprocesses geo-tagged Tweets, applies LDA, CTM, STM, and visualizes results.
result Automatic pooling of Tweets into pseudo-documents improves topic coherence.
New VAE models reveal hierarchical visual cortex computations.
problem Capturing hierarchical visual cortex computations in generative models.
method Sparse coding hierarchical VAEs trained on natural images with varied generative and recognition components.
result Representations similar to those in visual cortex emerge under inductive biases.
Visual system compares and evaluates machine learning models for clinical data predictions.
problem Challenges in comparing and evaluating different machine learning models for medical predictions.
method Developed a visual analytics system to compare and evaluate multiple models' prediction criteria and consistency.
result Demonstrated the effectiveness of the visual analytics system in assisting clinicians and researchers.
Interactive visualization helps understand complex machine learning models.
problem Low interpretability of machine learning models.
method Interactive slice visualization of predictor space, using interaction or touring algorithms.
result Enhances understanding and validation of machine learning model fits.
ARNe model excels in abstract visual reasoning tasks.
problem Abstract visual reasoning using attention mechanisms.
method Hybrid network architecture combining self-attention and relational reasoning.
result ARNe model surpasses WReN model by 11.28 ppt on PGM datasets.
A deep model generates diverse graph layouts.
problem Finding a good graph layout is time-consuming and haphazard.
method Deep generative model learns latent space for diverse layouts.
result Model learns and generalizes graph layout concepts.
High-contrast "gaudy" images improve the training of deep neural network models of visual cortexcs.CV
Gaudy images help train deep neural networks with less data.
problem Training deep neural networks with limited real data from visual cortex neurons.
method Used high-contrast binarized natural images (gaudy images) to train DNNs.
result Reduced training data needed for accurate DNN predictions of visual cortex neuron responses.
VTAB benchmarks diverse visual tasks to assess representation learning effectiveness.
problem Lack of a unified evaluation for general visual representations.
method Developed VTAB, a benchmark for diverse visual tasks, and evaluated many representation learning algorithms.
result VTAB revealed insights into the effectiveness of various representation learning methods.
Modern datasets and models are notoriously difficult to explore and analyze due to their inherent high dimensionality and massive numbers of samples. Existing visualization methods which employ dimensionality reduction to two or three dimensions are often inefficient and/or ineffective for these datasets. This paper in…