New SSIM measure improves adversarial attacks' perceptual quality.
problem Lack of perceptual quality in traditional adversarial attacks.
method Replaced Lp norms with SSIM measure for better quality. result SSIM-constrained attacks achieve better quality and similar success rate.
Stabilizes training of DNN for speech enhancement using PESQ scores.
problem Stability issues in training DNNs using non-differentiable OSQA scores.
method Approximate OSQA scores with a differentiable auxiliary DNN and stabilize training with reinforcement learning techniques.
result Stable training of DNN to achieve state-of-the-art PESQ scores and better sound quality.
Quaternion Conformer GAN (QC-GAN) is a parameter-efficient speech enhancement framework that combines a Quaternion Conformer generator with MetricGAN-based training.
problem Speech Enhancement
method Quaternion Conformer GAN
result Achieved a PESQ score of 3.48 with 0.89M parameters, comparable to state-of-the-art models at less than half their size.
Paper uses CNN to create realistic LiDAR data.
problem Creating high-resolution LiDAR data with missing points.
method Modified per-point loss and perceptual loss applied to generate realistic LiDAR data.
result Significant improvement in geometric and semantic accuracy over traditional methods.
In order to design haptic icons or build a haptic vocabulary, we require a set of easily distinguishable haptic signals to avoid perceptual ambiguity, which in turn requires a way to accurately estimate the perceptual (dis)similarity of such signals. In this work, we present a novel method to learn such a perceptual me…
This paper introduces a database to assess the perceptual similarity of adversarial images.
problem The lack of reliable metrics to assess the perceptual similarity of adversarial images generated by Lp norms. method Creation of a database and evaluation of fifteen FR image fidelity assessment metrics.
result The database and metrics can help in designing new metrics for adversarial examples.
PerceptNet mimics human vision to estimate image quality.
problem Estimating perceptual distance between images and their perturbations.
method Inspired by human visual system, PerceptNet uses convolutional neural network architecture.
result PerceptNet outperforms traditional image quality metrics and deep learning methods.
PeL separates sensory interface optimization from decision learning.
problem Optimizing sensory interfaces without task-specific information.
method Formal separation of perception and decision learning, using metrics for stability, informativeness, and geometry.
result Updates preserving invariants are orthogonal to decision gradients.
We introduce autoregressive implicit quantile networks (AIQN), a fundamentally different approach to generative modeling than those commonly used, that implicitly captures the distribution using quantile regression. AIQN is able to achieve superior perceptual quality and improvements in evaluation metrics, without incu…
Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
Paper proposes using NLPD for cGANs to improve image realism and segmentation.
problem Lack of perceptual evaluation in cGANs.
method Used NLPD to measure perceptual similarity, comparing with L1 distance.
result NLPD improves realism and segmentation accuracy in generated images.
New method disentangles perceptual uncertainty and behavioral costs in partially observable systems.
problem Tackles inverse optimal control for non-linear partially observable systems.
method Probabilistic approach using maximum causal entropy formulations and local linearization.
result Disentangles perceptual factors and behavioral costs in sequential decision-making.
Perceptual Kalman filters maintain human-perceptual quality while processing data.
problem Maintaining human-perceptual quality in signal processing under temporal constraints.
method An optimal causal filtering approach under a perfect perceptual-quality constraint.
result Adding perceptual quality constraints introduces a dilemma that requires sacrificing MSE for temporal consistency.
New finding: Perceptually-aligned gradients occur in adversarially-robust classifiers.
problem Understanding adversarial robustness in neural networks.
method Investigated perceptual alignment in adversarially-trained and smoothed classifiers.
result Perceptually-aligned gradients are a general property of robust classifiers.
PICLE uses probabilistic models to efficiently evaluate and compose modules for continual learning.
problem Challenging search space of module compositions in continual learning.
method Probabilistic framework to cheaply compute module compositions' fitness.
result First modular CL algorithm to achieve perceptual, few-shot, and latent transfer.
New method defends against unseen threat models using perceptual adversarial training.
problem Lack of precise mathematical characterization of human perception in adversarial attacks.
method Adversarial training against the set of all imperceptible adversarial examples approximated by deep neural networks.
result Perceptual Adversarial Training (PAT) achieves state-of-the-art robustness against multiple diverse adversarial attacks.
Study shows current metrics for audio adversarial examples are unreliable for human perception.
problem The reliability of metrics for evaluating audio adversarial examples.
method Analytical framework and human evaluation experiment.
result Current metrics for audio adversarial examples are not reliable for human perception.
Ordinal embedding methods estimate perceptual scales from relative judgments.
problem Measuring subjective sensation using relative judgments.
method Ordinal embedding from machine learning applied to method of triads.
result Ordinal embedding allows estimating perceptual scales from few judgments, non-monotonous functions, and multi-dimensional scales.
New hierarchical VQ-VAE scheme improves image compression quality and features at low bitrates.
problem Low bitrate image compression maintaining quality and features.
method Hierarchical VQ-VAE with stochastic quantization and Markovian latent variables.
result High perceptual quality and semantic features at low bitrates.
Study shows perceptual boost of visual attention varies with task difficulty and size.
problem Understanding how task-dependent the perceptual boost of visual attention is in natural settings.
method Designed and trained neural networks on various visual tasks, comparing results to baseline.
result Perceptual boost of attention is stronger with more difficult tasks and weaker with larger task sets.
Perceptual ad-blocking is a novel approach that detects online advertisements based on their visual content. Compared to traditional filter lists, the use of perceptual signals is believed to be less prone to an arms race with web publishers and ad networks. We demonstrate that this may not be the case. We describe att…
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
Paper proposes a method to make image model explanations robust to distortions.
problem Ensuring robustness of explanations for images under distortions.
method Embedding perceptual distances in surrogate explainers to evaluate and improve robustness.
result Surrogate explanations become more coherent and robust to distortions.
GAN normalizes CT scans for consistent radiomic feature values.
problem Variations in dose levels and slice thickness affect radiomic features sensitivity.
method Used a 3D generative adversarial network (GAN) to normalize reduced dose, thick slice images to normal dose, thinner slice images.
result GAN-based approach led to significantly smaller error in radiomic features.
A new perceptual adjustment query for metric learning reduces complexity in high-dimensional data.
problem Metric learning in high-dimensional data with limited human feedback.
method Inverted measurement scheme and two-stage estimator for PAQs.
result Sample complexity guarantees for the two-stage estimator of metric learning from PAQs.
Deep learning has brought an unprecedented progress in computer vision and significant advances have been made in predicting subjective properties inherent to visual data (e.g., memorability, aesthetic quality, evoked emotions, etc.). Recently, some research works have even proposed deep learning approaches to modify i…
Metric functions for phoneme perception capture the similarity structure among phonemes in a given language and therefore play a central role in phonology and psycho-linguistics. Various phenomena depend on phoneme similarity, such as spoken word recognition or serial recall from verbal working memory. This study prese…
Stochastic encoders outperform deterministic ones in 'perfect perceptual quality'.
problem Understanding when stochastic encoders outperform deterministic ones.
method Provided a toy example to illustrate performance.
result Stochastic encoders can significantly outperform deterministic ones in 'perfect perceptual quality'.
Conventional SVM-based image coding methods are founded on independently restricting the distortion in every image coefficient at some particular image representation. Geometrically, this implies allowing arbitrary signal distortions in an n-dimensional rectangle defined by the ε-insensitivity zone in eac…
Recent work has shown that additive threat models, which only permit the addition of bounded noise to the pixels of an image, are insufficient for fully capturing the space of imperceivable adversarial examples. For example, small rotations and spatial transformations can fool classifiers, remain imperceivable to human…
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
A new metric FRD improves comparing medical images.
problem Comparing medical images for distribution or domain differences.
method Developed a new metric FRD using standardized radiomic features.
result FRD outperforms other metrics in various medical imaging applications.
Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However, in recent years, it has become increasingly accepted that "low distortion" is no…
Learning a model of perceptual similarity from a collection of objects is a fundamental task in machine learning underlying numerous applications. A common way to learn such a model is from relative comparisons in the form of triplets: responses to queries of the form "Is object a more similar to b than it is to c?". I…
Improved image generation with private data using perceptual features.
problem Difficulty in training generative models with differential privacy.
method Use pre-trained perceptual features to learn private data distribution.
result Generative models can generate high-quality images with low privacy budget (ϵ≈2). Humans perceive the seemingly chaotic world in a structured and compositional way with the prerequisite of being able to segregate conceptual entities from the complex visual scenes. The mechanism of grouping basic visual elements of scenes into conceptual entities is termed as perceptual grouping. In this work, we pro…
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
A new loss function for VAEs improves image quality and efficiency.
problem Training VAEs to generate realistic images requires a loss function that reflects human perception.
method Based on Watson's perceptual model, the loss function computes a weighted distance in frequency space, accounts for luminance and contrast masking, and is extended to color images.
result VAEs trained with the new loss function generated high-quality, less blurry images with fewer artifacts and less computational resources.
Thanks to the growing availability of spoofing databases and rapid advances in using them, systems for detecting voice spoofing attacks are becoming more and more capable, and error rates close to zero are being reached for the ASVspoof2015 database. However, speech synthesis and voice conversion paradigms that are not…
Naive Bayes Nearest Neighbour (NBNN) is a simple and effective framework which addresses many of the pitfalls of K-Nearest Neighbour (KNN) classification. It has yielded competitive results on several computer vision benchmarks. Its central tenet is that during NN search, a query is not compared to every example in a d…
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
New benchmarks measure image generation models' ability to generalize beyond training data.
problem Trivially memorizing training data yields better scores than state-of-the-art models on current benchmarks.
method Developed neural network divergences (NNDs) as evaluation metrics requiring large samples.
result Implemented and validated a black-box metric that measures diversity, sample quality, and generalization.
GANs can be used to extract Fisher vectors for unsupervised feature learning.
problem Unsupervised feature extraction for classification and similarity tasks.
method Derive Fisher Information and Fisher Vectors from GANs' density model.
result GAN-induced Fisher Vectors perform competitively in unsupervised feature extraction.
A benchmark evaluates ioUS-to-MR synthesis methods for brain tumor surgery.
problem Difficult interpretation of ioUS images for brain tumor surgery.
method Six generators trained under four inference regimes and two targets on public data.
result SynDiff-2.5D best preserved downstream segmentation (U_Dice=0.55).
New method compresses facial videos using GANs and latent space optimization.
problem Efficiently compressing facial videos at low bit rates.
method Leverages StyleGAN for latent space representation and compression, learns optimal compression through entropy model and perceptual loss.
result Significantly reduces perceptual distortion at low bit rates compared to state-of-the-art codecs.
Many real world tasks such as reasoning and physical interaction require identification and manipulation of conceptual entities. A first step towards solving these tasks is the automated discovery of distributed symbol-like representations. In this paper, we explicitly formalize this problem as inference in a spatial m…
Fastens diffusion model sampling by distillation.
problem Slow sampling time of diffusion models.
method New parameterizations and progressive distillation.
result Models can be distilled to take half as many sampling steps.
The recent success of raw audio waveform synthesis models like WaveNet motivates a new approach for music synthesis, in which the entire process --- creating audio samples from a score and instrument information --- is modeled using generative neural networks. This paper describes a neural music synthesis model with fl…