A new perceptual adjustment query for metric learning reduces complexity in high-dimensional data.
problem Metric learning in high-dimensional data with limited human feedback.
method Inverted measurement scheme and two-stage estimator for PAQs.
result Sample complexity guarantees for the two-stage estimator of metric learning from PAQs.
Learning a model of perceptual similarity from a collection of objects is a fundamental task in machine learning underlying numerous applications. A common way to learn such a model is from relative comparisons in the form of triplets: responses to queries of the form "Is object a more similar to b than it is to c?". I…
Naive Bayes Nearest Neighbour (NBNN) is a simple and effective framework which addresses many of the pitfalls of K-Nearest Neighbour (KNN) classification. It has yielded competitive results on several computer vision benchmarks. Its central tenet is that during NN search, a query is not compared to every example in a d…
This paper generalizes graph representation for diverse data types.
problem Representing and querying hybrid data types in a unified way.
method Introducing a directed Tensor-Typed Multi-Graph with embeddings.
result Unified representation for visual, linguistic, and auditory data.
Locality sensitive hashing (LSH) is a powerful tool for sublinear-time approximate nearest neighbor search, and a variety of hashing schemes have been proposed for different dissimilarity measures. However, hash codes significantly depend on the dissimilarity, which prohibits users from adjusting the dissimilarity at q…
TQA improves prediction intervals for time series data by adjusting quantiles for both cross-sectional and longitudinal coverage.
problem Constructing reliable prediction intervals for cross-sectional time series data.
method Temporal Quantile Adjustment (TQA) method that adjusts the quantile in Conformal Prediction to account for both cross-sectional and longitudinal coverage.
result TQA improves longitudinal coverage while preserving cross-sectional coverage, as validated through extensive experimentation.
New SSIM measure improves adversarial attacks' perceptual quality.
problem Lack of perceptual quality in traditional adversarial attacks.
method Replaced Lp norms with SSIM measure for better quality. result SSIM-constrained attacks achieve better quality and similar success rate.
Perceptual Kalman filters maintain human-perceptual quality while processing data.
problem Maintaining human-perceptual quality in signal processing under temporal constraints.
method An optimal causal filtering approach under a perfect perceptual-quality constraint.
result Adding perceptual quality constraints introduces a dilemma that requires sacrificing MSE for temporal consistency.
New finding: Perceptually-aligned gradients occur in adversarially-robust classifiers.
problem Understanding adversarial robustness in neural networks.
method Investigated perceptual alignment in adversarially-trained and smoothed classifiers.
result Perceptually-aligned gradients are a general property of robust classifiers.
New method defends against unseen threat models using perceptual adversarial training.
problem Lack of precise mathematical characterization of human perception in adversarial attacks.
method Adversarial training against the set of all imperceptible adversarial examples approximated by deep neural networks.
result Perceptual Adversarial Training (PAT) achieves state-of-the-art robustness against multiple diverse adversarial attacks.
Paper introduces md-vtrees for efficient probabilistic and causal inference.
problem Efficient inference in complex probabilistic models.
method Introduces md-vtrees to generalize tractability conditions for advanced inference queries.
result Derives first polytime algorithms for causal inference queries.
Unified tractability conditions for various compositional inference queries.
problem Analyzing tractability of probabilistic and causal inference queries.
method Algebraic perspective on circuits, focusing on semiring operators.
result Unified sufficient conditions for tractable composition of operators.
We present a scalable, black box, perception-in-the-loop technique to find adversarial examples for deep neural network classifiers. Black box means that our procedure only has input-output access to the classifier, and not to the internal structure, parameters, or intermediate confidence values. Perception-in-the-loop…
Boomerang generates nonidentical images similar to input on image manifolds.
problem Generating nonidentical images similar to input on image manifolds.
method Adding noise to input image, moving closer to latent space, and mapping back through partial reverse diffusion.
result Boomerang generates nonidentical images similar to input on image manifolds.
Study shows perceptual boost of visual attention varies with task difficulty and size.
problem Understanding how task-dependent the perceptual boost of visual attention is in natural settings.
method Designed and trained neural networks on various visual tasks, comparing results to baseline.
result Perceptual boost of attention is stronger with more difficult tasks and weaker with larger task sets.
Perceptual ad-blocking is a novel approach that detects online advertisements based on their visual content. Compared to traditional filter lists, the use of perceptual signals is believed to be less prone to an arms race with web publishers and ad networks. We demonstrate that this may not be the case. We describe att…
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
In order to design haptic icons or build a haptic vocabulary, we require a set of easily distinguishable haptic signals to avoid perceptual ambiguity, which in turn requires a way to accurately estimate the perceptual (dis)similarity of such signals. In this work, we present a novel method to learn such a perceptual me…
Metric functions for phoneme perception capture the similarity structure among phonemes in a given language and therefore play a central role in phonology and psycho-linguistics. Various phenomena depend on phoneme similarity, such as spoken word recognition or serial recall from verbal working memory. This study prese…
PerceptNet mimics human vision to estimate image quality.
problem Estimating perceptual distance between images and their perturbations.
method Inspired by human visual system, PerceptNet uses convolutional neural network architecture.
result PerceptNet outperforms traditional image quality metrics and deep learning methods.
Paper tackles causal effect estimation in observational data with hidden variables.
problem Estimating causal effects in observational data with hidden confounders.
method Developed a theorem for local search to find superset of adjustment variables, proposing a data-driven algorithm.
result Proposed algorithm produces more accurate causal effect estimates than existing methods.
Stochastic encoders outperform deterministic ones in 'perfect perceptual quality'.
problem Understanding when stochastic encoders outperform deterministic ones.
method Provided a toy example to illustrate performance.
result Stochastic encoders can significantly outperform deterministic ones in 'perfect perceptual quality'.
QActor optimizes learning from noisy labeled data streams by querying experts for clean labels.
problem Learning from noisy labeled data in continuous streams with limited oracle queries.
method Combines quality models for filtering and oracle queries for true labels, dynamically adjusting query limits.
result QActor nearly matches optimal accuracy with up to 6% additional ground truth data from experts.
Conventional SVM-based image coding methods are founded on independently restricting the distortion in every image coefficient at some particular image representation. Geometrically, this implies allowing arbitrary signal distortions in an n-dimensional rectangle defined by the ε-insensitivity zone in eac…
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. Towards these capabilities, we present a probabilistic forecasting model of future interactions b…
Rate-In dynamically adjusts dropout rates during inference to improve uncertainty estimation in neural networks.
problem Static dropout rates lead to suboptimal uncertainty estimates in neural networks.
method Rate-In dynamically adjusts dropout rates using information-theoretic principles.
result Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates.
Recent work has shown that additive threat models, which only permit the addition of bounded noise to the pixels of an image, are insufficient for fully capturing the space of imperceivable adversarial examples. For example, small rotations and spatial transformations can fool classifiers, remain imperceivable to human…
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
P-BO reduces black-box adversarial attacks by 10x with Bayesian optimization and function prior.
problem Efficiently generating adversarial examples against black-box models.
method Prior-guided Bayesian Optimization (P-BO) with a function prior initialized from a surrogate model.
result Significantly reduces the number of queries needed for adversarial attacks.
Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However, in recent years, it has become increasingly accepted that "low distortion" is no…
Adaptive truncation improves privacy in online Bayesian estimation.
problem Ensuring privacy in online Bayesian estimation of a static parameter.
method Sequential Monte Carlo, adaptive truncation, Thompson sampling.
result Adaptive truncation reduces privacy-preserving noise, enabling more accurate estimation.
This paper presents a novel CNN-based approach for synthesizing high-resolution LiDAR point cloud data. Our approach generates semantically and perceptually realistic results with guidance from specialized loss-functions. First, we utilize a modified per-point loss that addresses missing LiDAR point measurements. Secon…
PeL separates sensory interface optimization from decision learning.
problem Optimizing sensory interfaces without task-specific information.
method Formal separation of perception and decision learning, using metrics for stability, informativeness, and geometry.
result Updates preserving invariants are orthogonal to decision gradients.
This paper will explore the use of autoencoders for semantic hashing in the context of Information Retrieval. This paper will summarize how to efficiently train an autoencoder in order to create meaningful and low-dimensional encodings of data. This paper will demonstrate how computing and storing the closest encodings…
FinMem enhances LLM trading agents with layered memory and character design.
problem Developing purpose-driven LLM agents for financial decision-making.
method Integrates layered memory and character design modules into an LLM framework.
result Significantly enhanced trading performance in financial markets.
Improved image generation with private data using perceptual features.
problem Difficulty in training generative models with differential privacy.
method Use pre-trained perceptual features to learn private data distribution.
result Generative models can generate high-quality images with low privacy budget (ϵ≈2). b-LOAD extends local causal discovery with prior knowledge, improving causal effect estimation.
problem Local causal discovery struggles in data-scarce settings due to uncertainty and incomplete neighborhoods.
method b-LOAD incorporates prior knowledge directly into local structure learning, using Meek's rules to refine discovery.
result b-LOAD refines the admissible equivalence class and enlarges identifiable causal queries, improving causal effect estimation.
New method disentangles perceptual uncertainty and behavioral costs in partially observable systems.
problem Tackles inverse optimal control for non-linear partially observable systems.
method Probabilistic approach using maximum causal entropy formulations and local linearization.
result Disentangles perceptual factors and behavioral costs in sequential decision-making.
Stabilizes training of DNN for speech enhancement using PESQ scores.
problem Stability issues in training DNNs using non-differentiable OSQA scores.
method Approximate OSQA scores with a differentiable auxiliary DNN and stabilize training with reinforcement learning techniques.
result Stable training of DNN to achieve state-of-the-art PESQ scores and better sound quality.
Humans perceive the seemingly chaotic world in a structured and compositional way with the prerequisite of being able to segregate conceptual entities from the complex visual scenes. The mechanism of grouping basic visual elements of scenes into conceptual entities is termed as perceptual grouping. In this work, we pro…
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
A new loss function for VAEs improves image quality and efficiency.
problem Training VAEs to generate realistic images requires a loss function that reflects human perception.
method Based on Watson's perceptual model, the loss function computes a weighted distance in frequency space, accounts for luminance and contrast masking, and is extended to color images.
result VAEs trained with the new loss function generated high-quality, less blurry images with fewer artifacts and less computational resources.
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
Improved few-shot learning with LSSVM and transductive modules.
problem Few-shot learning with limited data and samples.
method Introducing LSSVM as a base learner and transductive modules to enhance classification accuracy.
result FSLSTM achieves state-of-the-art performance on miniImageNet and CIFAR-FS benchmarks.
Adversarial attacks on deep learning models have compromised their performance considerably. As remedies, a lot of defense methods were proposed, which however, have been circumvented by newer attacking strategies. In the midst of this ensuing arms race, the problem of robustness against adversarial attacks still remai…
The paper studies a method to sample nodes from a massive graph using personalized PageRank.
problem Sampling from a massive network is expensive and impractical; the paper provides an alternative.
method The paper introduces a crawling method to approximate the personalized PageRank vector without querying the entire graph.
result The adjusted personalized PageRank vector can effectively select nodes within the same block as the seed node.
Quaternion Conformer GAN (QC-GAN) is a parameter-efficient speech enhancement framework that combines a Quaternion Conformer generator with MetricGAN-based training.
problem Speech Enhancement
method Quaternion Conformer GAN
result Achieved a PESQ score of 3.48 with 0.89M parameters, comparable to state-of-the-art models at less than half their size.