Contrastive learning struggles with class collapse and feature suppression, revealing bias towards simpler solutions.
problem Contrastive learning struggles with class collapse and feature suppression, especially in supervised and unsupervised settings.
method Unified theoretical framework to determine which features are learnt by CL, revealing bias towards simpler solutions.
result Bias towards simpler solutions is a key factor in class collapse and feature suppression.
Contrastive learning properties studied, including feature suppression and hierarchical learning.
problem Feature suppression and hierarchical learning in contrastive learning.
method Generalized contrastive loss, instance-based contrastive learning, explicit and controllable competing features.
result Contrastive learning can suppress and prevent the learning of competing features.
Despite the remarkable performance of deep neural networks on various computer vision tasks, they are known to be susceptible to adversarial perturbations, which makes it challenging to deploy them in real-world safety-critical applications. In this paper, we conjecture that the leading cause of adversarial vulnerabili…
Regularization methods can overregulate, suppressing causal features.
problem Mitigating shortcuts in models exploiting spurious correlations.
method Analysis of regularization methods and their effects on causal features.
result Regularization can overregulate, suppressing causal features.
Study how models represent features in naturalistic learning problems.
problem Understanding which features models use and ignore in naturalistic tasks.
method Synthetic datasets with controlled task-relevance of features, training models to recognize both easy and hard features.
result Models preferentially represent task-relevant features and suppress task-irrelevant ones over training.
A new method suppresses echo more effectively with lower latency.
problem Echo cannot be fully removed by linear filters due to nonlinear relationships.
method Uses a modified Conv-TasNet with multiple streams of input signals.
result Efficacy validated in both single-talk and double-talk situations.
Paper proposes a statistical model for detecting mu-suppression in EEG signals.
problem Detecting mu-suppression in motor imagery EEG signals.
method Proposes a statistical model based on the generalized extreme value distribution (GEV) and a linear classifier.
result Preliminary results show good classification accuracy in detecting mu-suppression and distinguishing EEG events.
LLMs prefer Bitcoin under crisis frames, affecting financial decisions.
problem Testing whether LLMs have built-in biases towards specific financial assets.
method Developed a three-level audit protocol to examine Bitcoin's representation and influence in LLMs.
result An identifiable internal feature in LLMs can be perturbed to move financial choices, but only within measurable limits.
Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.
problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.
method Conditional Gaussian representations and SDE system with decoupled backward weights.
result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.
Deep-neural-network (DNN) based noise suppression systems yield significant improvements over conventional approaches such as spectral subtraction and non-negative matrix factorization, but do not generalize well to noise conditions they were not trained for. In comparison to DNNs, humans show remarkable noise suppress…
We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process. Specifically, we introduce an asymmetric autoencoder term that allows reliable predictors fo…
Stochastic gradient descent updates parameters with summation gradient computed from a random data batch. This summation will lead to unbalanced training process if the data we obtained is unbalanced. To address this issue, this paper takes the error variance and error mean both into consideration. The adaptively adjus…
Suppressing bones on chest X-rays such as ribs and clavicle is often expected to improve pathologies classification. These bones can interfere with a broad range of diagnostic tasks on pulmonary disease except for musculoskeletal system. Current conventional method for acquisition of bone suppressed X-rays is dual ener…
PatternLocal improves XAI for non-linear models by suppressing suppressor variables.
problem Suppressor variables cause false-positive feature attributions in non-linear models.
method PatternLocal uses locally linear surrogate models and transforms weights into a generative representation.
result PatternLocal reduces false-positive attributions and provides more reliable explanations.
Adversarial training has been successfully applied to build robust models at a certain cost. While the robustness of a model increases, the standard classification accuracy declines. This phenomenon is suggested to be an inherent trade-off. We propose a model that employs feature prioritization by a nonlinear attention…
Paper accelerates K-means clustering for large sparse document data.
problem Efficiently clustering large-scale sparse document data.
method Designs an AFM algorithm leveraging UCs and inverted-index structure.
result Significantly improves clustering speed for large-scale documents.
Unified framework suppresses model bias in semi-supervised learning with decoupled sampling control.
problem Class imbalance in semi-supervised learning, especially with distributional mismatches.
method Unified framework SC-SSL with decoupled sampling control, explicit expansion capability, and adaptive sampling probabilities.
result Consistent and state-of-the-art performance across various benchmark datasets and distribution settings.
This paper proposes a novel selective autoencoder approach within the framework of deep convolutional networks. The crux of the idea is to train a deep convolutional autoencoder to suppress undesired parts of an image frame while allowing the desired parts resulting in efficient object detection. The efficacy of the fr…
CNNs can develop blind spots due to uneven padding in feature maps.
problem Spatial bias in convolutional networks leads to blind spots in certain tasks.
method Identified and analyzed the role of padding in convolutional networks, proposing solutions to mitigate bias.
result Mitigating spatial bias improves model accuracy, especially in tasks like small object detection.
DG separates successes and failures by gating updates with advantage and surprisal.
problem Negative learning from surprising data in distributed reinforcement learning.
method DG gates each update with the product of advantage and surprisal, suppressing failures and preserving successes.
result DG outperforms other methods in various challenging reinforcement learning tasks.
Acoustic scenes are rich and redundant in their content. In this work, we present a spatio-temporal attention pooling layer coupled with a convolutional recurrent neural network to learn from patterns that are discriminative while suppressing those that are irrelevant for acoustic scene classification. The convolutiona…
New method improves fairness in DP learning by preventing excessive gradient suppression.
problem Disparate impact on model predictions for minority groups in DP learning.
method Bounded adaptive clipping to prevent excessive gradient suppression.
result Improves worst-class accuracy by over 10 percentage points compared to existing methods.
This paper addresses image classification through learning a compact and discriminative dictionary efficiently. Given a structured dictionary with each atom (columns in the dictionary matrix) related to some label, we propose cross-label suppression constraint to enlarge the difference among representations for differe…
Theory explains generalization in deep learning, reducing memorization and improving performance.
problem Understanding and improving generalization in deep learning models.
method Developed a non-asymptotic theory using the empirical neural tangent kernel.
result Generalization is possible even when the kernel evolves significantly, with coherent signal accumulation and noise suppression.
AEN-SAEs address feature starvation in sparse autoencoders by stabilizing the geometric alignment of sparse coding.
problem Feature starvation in sparse autoencoders, leading to unstable and misaligned representations.
method Adaptive Elastic Net SAEs (AEN-SAEs) combine ℓ2 and ℓ1 terms to stabilize the sparse coding map and control feature interactions. result AEN-SAEs mitigate feature starvation without heuristic resampling, maintaining competitive reconstruction abilities.
Cloak identifies essential features to preserve prediction privacy without provider collaboration.
problem Discovering the subset of features necessary for a prediction task.
method Gradient-based perturbation maximization method to identify essential features, followed by suppression of the rest using utility-preserving constant values.
result Cloak reduces mutual information between input and sifted representations by 85.01% with negligible utility loss.
Localized Multidirectional Correction improves non-refusal target-response behavior in foundation models.
problem Controlled post-training refusal suppression in routed MoE and hybrid-MoE foundation models.
method Introduce Localized Multidirectional Correction (LoMC), a support-gated intervention framework.
result Substantially improves non-refusal target-response behavior while maintaining general capability under a compact intervention footprint.
Recurrent neural networks are known for their notorious exploding and vanishing gradient problem (EVGP). This problem becomes more evident in tasks where the information needed to correctly solve them exist over long time scales, because EVGP prevents important gradient components from being back-propagated adequately …
SGD quickly learns a spurious XOR feature before the signal feature, revealing learning dynamics.
problem Over-reliance on spurious correlations in neural networks trained by SGD.
method Theoretical analysis of SGD on two-layer ReLU networks trained on XOR data.
result SGD learns the spurious feature first and exponentially fast, dominating the signal feature.
New method uses random features and Tikhonov regularization for operator learning from noisy data.
problem Accurate approximation of mappings between infinite-dimensional function spaces with reduced training time.
method Regularized random Fourier features (RRFF) coupled with finite element reconstruction (RRFF-FEM).
result The method achieves improved performance with reduced training time and noise robustness.
Bayesian method suppresses low-frequency pulses in audio recordings.
problem Suppressing long pulses caused by mechanical defects in audio recordings.
method Bayesian approach using Gaussian Process for pulse location, signal interpolation, and tail estimation.
result Perceptual results similar to previous methods, performs well on naturally degraded signals.
Paper proposes a new method to optimize feature coordinates for better image classification.
problem Improving feature extraction for better machine learning classification.
method Mutual-energy inner product optimization method.
result The method enhances low-frequency features and suppresses high-frequency noise, leading to better classification results.
Point clouds, as a form of Lagrangian representation, allow for powerful and flexible applications in a large number of computational disciplines. We propose a novel deep-learning method to learn stable and temporally coherent feature spaces for points clouds that change over time. We identify a set of inherent problem…
Improved market state classification for risk assessment.
problem Classifying financial market states for better risk assessment.
method Modified selection criteria for market states, clustering optimization, and visualization of correlation matrices.
result Statistically significant results in SP 500 and Nikkei 225 markets.
DDGM generates realistic ECG signals for clinical use.
problem Generating accurate ECG signals from noisy data.
method Bayesian ECG reconstruction using DDGM trained on healthy ECG data.
result DDGM successfully generates realistic ECG signals for clinical applications.
Unified framework for removing unwanted information from machine learning models.
problem Removing undesirable features or data points from machine learning models while preserving utility.
method Information-theoretic regularization approach for data point and feature unlearning.
result Unified mathematical framework with provable guarantees for both data point and feature unlearning.
RoGAT enhances GAT robustness against adversarial attacks.
problem Vulnerability of GAT to adversarial attacks.
method Dynamic adjustment of edge weights and features, with an extra attention score.
result RoGAT outperforms other defensive methods in robustness tests.
We argue that the existing regret matchings for Nash equilibrium approximation conduct "jumpy" strategy updating when the probabilities of future plays are set to be proportional to positive regret measures. We propose a geometrical regret matching which features "smooth" strategy updating. Our approach is simple, intu…
For data sets populated by a very well modeled process and by another process of unknown probability density function (PDF), a desired feature when manipulating the fraction of the unknown process (either for enhancing it or suppressing it) consists in avoiding to modify the kinematic distributions of the well modeled …
Consistency distillation reduces memorization in diffusion models without harming sample quality.
problem Understanding how distillation affects memorization in diffusion models.
method Analysis of consistency distillation in diffusion models using a random feature neural network model.
result Consistency distillation reduces memorization in diffusion models without harming sample quality.
HIV RNA viral load (VL) is an important outcome variable in studies of HIV infected persons. There exists only a handful of methods which classify patients by viral load patterns. Most methods place limits on the use of viral load measurements, are often specific to a particular study design, and do not account for com…
Deep neural networks (DNNs) are increasingly powering high-stakes applications such as autonomous cars and healthcare; however, DNNs are often treated as "black boxes" in such applications. Recent research has also revealed that DNNs are highly vulnerable to adversarial attacks, raising serious concerns over deploying …
Proposes TFCL to mitigate negative transfer in MTL by collaborating across features and tasks.
problem Negative transfer in Multi-Task Learning (MTL) due to dissimilar tasks.
method Task-Feature Collaborative Learning (TFCL) with heterogeneous block-diagonal structure regularizer and optimization method.
result Global convergence and block-diagonal structure recovery guarantees.
New methods validate a hypothesis explaining how neural nets generalize well.
problem Why over-parameterized nets generalize well despite memorizing training data.
method Developed new algorithms to suppress weak gradient directions without per-example gradients.
result Validated a hypothesis about gradient directions and their role in generalization.
Adversarial domain-invariant training (ADIT) proves to be effective in suppressing the effects of domain variability in acoustic modeling and has led to improved performance in automatic speech recognition (ASR). In ADIT, an auxiliary domain classifier takes in equally-weighted deep features from a deep neural network …
SNAP improves robust computation by emphasizing trustworthy items and downweighting outliers.
problem Improving robustness in computation, especially in high-dimensional settings.
method SNAP assigns weights based on mutual agreement, suppressing outlier contributions.
result SNAP ensures outliers contribute negligibly to computations, even in high-dimensional settings.
M2M tackles zero-shot structured noise suppression in images.
problem Structured noise with strong anisotropic correlations in real-world images.
method M2M introduces a novel sampling strategy that generates pseudo-independent sub-image pairs from a single noisy input, using directional interpolation and generalized median filtering.
result M2M consistently outperforms state-of-the-art zero-shot methods under correlated noise.
TSInsight improves interpretability of deep time-series models.
problem Lack of interpretability methods for time-series data.
method Attach auto-encoder to classifier with sparsity-inducing norm, fine-tune based on gradients and reconstruction penalty.
result TSInsight effectively boosts interpretability of deep time-series models.