PerceptNet mimics human vision to estimate image quality.
problem Estimating perceptual distance between images and their perturbations.
method Inspired by human visual system, PerceptNet uses convolutional neural network architecture.
result PerceptNet outperforms traditional image quality metrics and deep learning methods.
New method disentangles perceptual uncertainty and behavioral costs in partially observable systems.
problem Tackles inverse optimal control for non-linear partially observable systems.
method Probabilistic approach using maximum causal entropy formulations and local linearization.
result Disentangles perceptual factors and behavioral costs in sequential decision-making.
Enhanced text-to-speech synthesizes expressive speech from a single example.
problem Creating a new expressive speech style from a single example of speech.
method Combines VAE and Normalizing Flows to improve disentanglement and naturalness.
result Reduces KL-divergence by 22% and improves perceptual metrics.
Study neural systems' classification of perceptual manifolds.
problem Classifying neural responses to diverse sensory features.
method Statistical mechanics theory for linear classification of manifolds.
result Introduction of novel geometrical measures for manifold analysis.
Paper introduces adversarial lossy compression for video artifacts reduction.
problem Unpleasant reconstruction artifacts in standard video coding schemes at low bit-rates.
method Adversarial lossy video compression model minimizing an adversarial distortion objective.
result Reduction of perceptual artifacts and detail reconstruction under extreme compression.
New SSIM measure improves adversarial attacks' perceptual quality.
problem Lack of perceptual quality in traditional adversarial attacks.
method Replaced Lp norms with SSIM measure for better quality. result SSIM-constrained attacks achieve better quality and similar success rate.
New method creates imperceptible and robust adversarial examples.
problem Vulnerability of neural networks to adversarial examples.
method Human perceptual system consideration and maximizing noise tolerance.
result Efficacy demonstrated through experimental results.
Perceptual Kalman filters maintain human-perceptual quality while processing data.
problem Maintaining human-perceptual quality in signal processing under temporal constraints.
method An optimal causal filtering approach under a perfect perceptual-quality constraint.
result Adding perceptual quality constraints introduces a dilemma that requires sacrificing MSE for temporal consistency.
Spatially transformed adversarial examples are perceptually realistic and harder to defend against.
problem Vulnerability of deep neural networks to adversarial examples.
method Spatial transformation of images to generate adversarial examples.
result Spatially transformed adversarial examples are more difficult to defend against existing methods.
Neural networks trained on natural scenes exhibit Gestalt closure.
problem Understanding how visual perception groups elements in natural scenes.
method Used deep-learning methods to investigate natural scene statistics and their relation to Gestalt laws.
result A neural network trained on natural images exhibits closure on synthetic edge fragment displays.
New finding: Perceptually-aligned gradients occur in adversarially-robust classifiers.
problem Understanding adversarial robustness in neural networks.
method Investigated perceptual alignment in adversarially-trained and smoothed classifiers.
result Perceptually-aligned gradients are a general property of robust classifiers.
New method defends against unseen threat models using perceptual adversarial training.
problem Lack of precise mathematical characterization of human perception in adversarial attacks.
method Adversarial training against the set of all imperceptible adversarial examples approximated by deep neural networks.
result Perceptual Adversarial Training (PAT) achieves state-of-the-art robustness against multiple diverse adversarial attacks.
Perceptual ad-blocking is vulnerable to attacks, creating new security risks.
problem Vulnerability of perceptual ad-blocking to attacks and new security risks.
method Analysis and creation of adversarial examples to bypass perceptual ad-blocking.
result Perceptual ad-blocking can be bypassed using adversarial examples, introducing new security risks.
Study shows perceptual boost of visual attention varies with task difficulty and size.
problem Understanding how task-dependent the perceptual boost of visual attention is in natural settings.
method Designed and trained neural networks on various visual tasks, comparing results to baseline.
result Perceptual boost of attention is stronger with more difficult tasks and weaker with larger task sets.
This work introduces a new threat model for adversarial attacks using perceptual metrics.
problem Insufficient threat models for capturing imperceivable adversarial examples.
method Leverage quantitative perceptual metrics (LPIPS, SSIM) to define a novel threat model.
result Combined attacks retain perceptual distortion but induce higher misclassification rates.
PerceptNet learns haptic signal similarity using human data.
problem Designing haptic icons requires accurate perceptual similarity estimation.
method Deep neural network projecting signals to an embedding space with a triplet loss.
result Our method effectively models perceptual dissimilarity compared to alternatives.
Human cochlear models improve DNN noise suppression systems.
problem DNN-based noise suppression systems lack robustness to unseen noise conditions.
method Coupled cochlear models with DNNs to improve noise suppression.
result Cochlear models enhance DNN generalizability to various noise conditions.
New framework learns phoneme metric from perception data.
problem Learn metric for phoneme similarity from data.
method Learning algorithms to derive metric from behavioral data.
result Framework outperforms previous metrics in phoneme prediction.
Introduces PCG for better counterfactual explanations in vision models.
problem Ambiguity in latent-space optimization methods for counterfactual explanations.
method Constructs counterfactuals by tracing geodesics under a perceptually Riemannian metric.
result PCG outperforms baselines and reveals hidden failure modes.
New model separates object attributes for better perceptual grouping.
problem Perceptual grouping of complex visual scenes.
method Spatial mixture models with learnable priors.
result Outperforms state-of-the-art methods in perceptual grouping.
Paper uses CNN to create realistic LiDAR data.
problem Creating high-resolution LiDAR data with missing points.
method Modified per-point loss and perceptual loss applied to generate realistic LiDAR data.
result Significant improvement in geometric and semantic accuracy over traditional methods.
A new perceptual adjustment query for metric learning reduces complexity in high-dimensional data.
problem Metric learning in high-dimensional data with limited human feedback.
method Inverted measurement scheme and two-stage estimator for PAQs.
result Sample complexity guarantees for the two-stage estimator of metric learning from PAQs.
New study shows tradeoffs between compression quality, distortion, and perception.
problem Optimizing compression for low distortion often sacrifices perceptual quality.
method Adopted Blau & Michaeli's perceptual quality definition and studied the rate-distortion-perception tradeoff.
result Restricting perceptual quality to high generally requires a trade-off between rate and distortion.
End-to-end method for robot localization in simulated and real environments.
problem Generating robot actions to maximize pose disambiguation in a reference map.
method Differentiable learning of perception and planning modules, using convolutional neural networks and deep reinforcement learning.
result The system outperforms traditional approaches for perception or planning.
GAN normalizes CT scans for consistent radiomic feature values.
problem Variations in dose levels and slice thickness affect radiomic features sensitivity.
method Used a 3D generative adversarial network (GAN) to normalize reduced dose, thick slice images to normal dose, thinner slice images.
result GAN-based approach led to significantly smaller error in radiomic features.
Adversarial fog tests autonomous navigation models.
problem Neural networks are fooled by adversarial perturbations, but fog naturally creates similar perturbations.
method Introduced a new type of adversarial perturbation using generative models and Cycle-Consistent Generative Adversarial Networks.
result Generated adversarial fog images help test autonomous navigation models.
Study improves low-quality speech data for voice cloning.
problem Feasibility of training spoofing systems with low-quality data.
method Developed a GAN-based speech enhancement system, trained TTS and voice conversion models.
result Significant improvement in SNR and perceptual cleanliness of low-quality data.
Stochastic encoders outperform deterministic ones in 'perfect perceptual quality'.
problem Understanding when stochastic encoders outperform deterministic ones.
method Provided a toy example to illustrate performance.
result Stochastic encoders can significantly outperform deterministic ones in 'perfect perceptual quality'.
Conventional SVM-based image coding methods are founded on independently restricting the distortion in every image coefficient at some particular image representation. Geometrically, this implies allowing arbitrary signal distortions in an n-dimensional rectangle defined by the ε-insensitivity zone in eac…
Neural network VQ-VAE with WaveNet decodes speech at 1.6 kbps with high quality.
problem Efficiently transmitting and storing speech signals at low bit-rates.
method VQ-VAE and WaveNet architecture for speech coding.
result Speech coding at 1.6 kbps with perceptual quality between MELP and AMR-WB.
Children learn concepts without explicit teaching by aligning internal systems.
problem Learning concepts from noisy perceptual experience without explicit teaching.
method Using information in the environment to build and align conceptual systems.
result The more concepts and systems there are, the easier learning becomes.
PeL separates sensory interface optimization from decision learning.
problem Optimizing sensory interfaces without task-specific information.
method Formal separation of perception and decision learning, using metrics for stability, informativeness, and geometry.
result Updates preserving invariants are orthogonal to decision gradients.
A detector subnetwork improves deep learning robustness against adversarial perturbations.
problem Vulnerability of deep neural networks to adversarial perturbations that are imperceptible to humans.
method Augmenting deep neural networks with a binary classification detector subnetwork trained to distinguish adversarial from genuine data.
result Adversarial perturbations can be detected surprisingly well even though they are imperceptible to humans.
Improved image generation with private data using perceptual features.
problem Difficulty in training generative models with differential privacy.
method Use pre-trained perceptual features to learn private data distribution.
result Generative models can generate high-quality images with low privacy budget (ϵ≈2). Paper proposes using NLPD for cGANs to improve image realism and segmentation.
problem Lack of perceptual evaluation in cGANs.
method Used NLPD to measure perceptual similarity, comparing with L1 distance.
result NLPD improves realism and segmentation accuracy in generated images.
Stabilizes training of DNN for speech enhancement using PESQ scores.
problem Stability issues in training DNNs using non-differentiable OSQA scores.
method Approximate OSQA scores with a differentiable auxiliary DNN and stabilize training with reinforcement learning techniques.
result Stable training of DNN to achieve state-of-the-art PESQ scores and better sound quality.
Photo-realistic super-resolution using GANs for large upscaling factors.
problem Recovering fine texture details at large upscaling factors.
method SRGAN, a GAN framework with adversarial and content losses.
result Significantly improved perceptual quality compared to state-of-the-art methods.
PCI combines perception and control using Bayesian inference with object-based representations.
problem Separate perception and control in reinforcement learning.
method Joint Perception and Control as Inference (PCI) framework with Object-based Perception Control (OPC).
result OPC achieves good perceptual grouping quality and outperforms baselines in accumulated rewards.
A new loss function for VAEs improves image quality and efficiency.
problem Training VAEs to generate realistic images requires a loss function that reflects human perception.
method Based on Watson's perceptual model, the loss function computes a weighted distance in frequency space, accounts for luminance and contrast masking, and is extended to color images.
result VAEs trained with the new loss function generated high-quality, less blurry images with fewer artifacts and less computational resources.
Paper presents a new video generation model using diffusion probabilistic methods.
problem Generating high-quality video sequences.
method Denoising diffusion probabilistic models, autoregressive, end-to-end optimization.
result Significant improvements in perceptual quality and probabilistic frame forecasting.
This paper introduces a database to assess the perceptual similarity of adversarial images.
problem The lack of reliable metrics to assess the perceptual similarity of adversarial images generated by Lp norms. method Creation of a database and evaluation of fifteen FR image fidelity assessment metrics.
result The database and metrics can help in designing new metrics for adversarial examples.
Quaternion Conformer GAN (QC-GAN) is a parameter-efficient speech enhancement framework that combines a Quaternion Conformer generator with MetricGAN-based training.
problem Speech Enhancement
method Quaternion Conformer GAN
result Achieved a PESQ score of 3.48 with 0.89M parameters, comparable to state-of-the-art models at less than half their size.
New model explains music emotion predictions with visualizable mid-level features.
problem Challenging to quantify and predict music emotion.
method VGG-style deep neural network with mid-level perceptual features.
result Small loss in performance justifies high explainability of predictions.
ExGANs use exemplar information to improve eye in-painting results.
problem Improving photo-realistic eye in-painting results.
method Conditional GAN with exemplar information at multiple points.
result ExGANs produce photo-realistic, personalized in-painting results.
New method compresses facial videos using GANs and latent space optimization.
problem Efficiently compressing facial videos at low bit rates.
method Leverages StyleGAN for latent space representation and compression, learns optimal compression through entropy model and perceptual loss.
result Significantly reduces perceptual distortion at low bit rates compared to state-of-the-art codecs.
Ordinal embedding methods estimate perceptual scales from relative judgments.
problem Measuring subjective sensation using relative judgments.
method Ordinal embedding from machine learning applied to method of triads.
result Ordinal embedding allows estimating perceptual scales from few judgments, non-monotonous functions, and multi-dimensional scales.
This paper re-evaluates TD in deep RL, finding MC can be a viable alternative.
problem Understanding the role of temporal differencing (TD) in deep reinforcement learning.
method Designed environments to control for factors affecting performance in deep RL, comparing TD with infinite-horizon Monte Carlo (MC).
result Finite-horizon Monte Carlo is not inferior to TD, even with sparse or delayed rewards.
Inspired by brain's modality fusion, this paper detects active speakers from audio and video.
problem Detecting active speakers in noisy environments.
method Inspired by brain's superior colliculus, combines audio and visual data through specialized neural networks and a novel fusion layer.
result Achieved results greatly surpassing initial expectations, confirming the effectiveness of the proposed method.