Stabilizes deep Bayesian neural networks with self-stabilizing priors.
problem Brittleness and difficulty in training deep Bayesian neural networks.
method Signal propagation theory, reformulated ELBO, self-stabilizing priors.
result Improved convergence and robustness in training deeper networks and noisier settings.
Paper proposes a new framework to improve stability-based bounds in deep learning.
problem Explaining generalization in overparameterized neural networks.
method Decomposes excess risk dynamics into signal and noise components, applying stability-based bounds only to the noise.
result The decomposition framework improves stability-based bounds and explains generalization in neural networks.
Improves CNN stability by translating classical signal denoising methods.
problem Stability of CNNs is poorly understood.
method Interprets classical signal denoising methods as ResNet architectures.
result Translates diffusivities, shrinkage functions, and regularizers into CNN activation functions.
Backpropagation-free RL method trains layers using local signals.
problem Vanishing or exploding gradients in backpropagation-based RL.
method Local pairwise distance matching for layer-wise training without backpropagation.
result Backpropagation-free method achieves competitive performance and stability.
The paper analyzes deep neural networks using rectified linear units.
problem Understanding the individual affine linear representations of deep neural networks.
method Signal processing perspective, atomic decompositions, Lipschitz regularity estimation.
result Conditions for stabilizing learning in deep neural networks without network depth constraints.
GNNs improve graph signal discrimination by adding nonlinearities.
problem Improving graph signal discrimination in physical networks.
method Analyzing the discriminability of GNNs and their relation to graph filter banks.
result GNNs are at least as discriminative as linear graph filter banks.
Stable algebraic filters improve neural network performance.
problem Improving neural network stability to deformations.
method Analyzed stability of algebraic filters and neural networks under deformations of the homomorphism.
result Stable algebraic filters have frequency responses whose derivative is inversely proportional to frequency.
Wiatowski and Bölcskei, 2015, proved that deformation stability and vertical translation invariance of deep convolutional neural network-based feature extractors are guaranteed by the network structure per se rather than the specific convolution kernels and non-linearities. While the translation invariance result appli…
GNNs generalize CNNs for graph data, showing equivariance and stability.
problem Processing signals on graphs.
method Graph convolutional filters, nonlinearities, stacked layers.
result GNNs converge to graphon neural networks under graph convergence.
Despite their impressive performance, deep convolutional neural networks (CNNs) have been shown to be sensitive to small adversarial perturbations. These nuisances, which one can barely notice, are powerful enough to fool sophisticated and well performing classifiers, leading to ridiculous misclassification results. In…
GNNs maintain stability under minor graph topology changes.
problem Stability of GNNs to small graph topology changes.
method Proved stability of GNNs with integral Lipschitz filters.
result GNNs output change is bounded by relative graph topology change.
New BMI interface stabilizes user experience over long periods.
problem Inconsistent neural signal data leads to interface recalibration.
method Adversarial Domain Adaptation Network to match residual distributions.
result Adversarial Domain Adaptation Network outperforms other methods.
Paper proves spectral filters can be transferred between graphs.
problem Proving spectral filters can be transferred between graphs.
method Introducing the Cayley smoothness space and proving filters in this space are linearly stable.
result Graph spectral filters are transferable if they are in the Cayley smoothness space.
The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals. However, a precise study of these properties and how they affect learning guarantees is still missing. In this paper, we consider deep convolutional represen…
New approach uses neural networks for MIMO modulation without black-box optimization.
problem Improving MIMO modulation for non-coherent channels with short coherence windows.
method Simulation-driven optimization using neural networks for modulation and signal detection design.
result Demonstrated that neural networks can be avoided while still achieving comparable performance in MIMO communications.
Complex-valued neural networks improve seismic data analysis by preserving phase information.
problem Low-frequency aliasing in seismic data due to discarded phase information.
method Developed complex-valued deep convolutional networks to leverage phase information in deterministic physical data.
result Complex-valued networks outperform real-valued networks in training and inference from deterministic physical data.
Graph neural networks leverage graph filters to learn from network data.
problem Learning from network data with graph structure.
method Characterize graph neural networks using graph signal processing and graph convolutional filters.
result Graph neural networks have permutation equivariance and stability to topology changes.
Paper tackles GAN instability in audio and speech signals using a new similarity metric.
problem Improving stability of LS-GANs for audio and speech signals.
method Proposes a new similarity metric in unitary space of Schur decomposition for 2D audio and speech representations.
result Enhanced stability in training with less mode collapse compared to baseline GANs.
A method uses non-autonomous equations to classify time signals efficiently.
problem Time signal classification with minimal parameters and high accuracy.
method Develops a framework using non-autonomous dynamical equations to classify time signals.
result The method achieves comparable accuracy with fewer parameters than existing methods.
Improved speech separation and enhancement using neural beamforming.
problem Challenging speech separation and enhancement in reverberant environments.
method Sequential neural beamforming combining spectral and spatial separation methods.
result Average improvement of 2.75 dB in scale-invariant signal-to-noise ratio and 14.2% absolute reduction in speech recognition metric.
New empirical index reveals wider validity of Echo State Property in input-driven reservoirs.
problem Lack of proper input consideration in Echo State Property conditions.
method Introduced an empirical Echo State Property index to analyze stability of reservoirs with input signals.
result The actual domain of Echo State Property validity is wider than literature conditions suggest.
Outliers with opposing signals significantly affect neural network optimization.
problem Understanding and mitigating the impact of outliers with opposing signals on neural network training.
method Identifying and analyzing pairs of outliers with strong opposing signals in training data.
result Outliers with opposing signals can cause optimization to enter a narrow valley, leading to oscillatory behavior and eventual loss spikes.
ERNN improves RNN accuracy and stability with time-delayed self-feedback.
problem Inaccuracy and instability in RNNs.
method Augmenting RNN with a time-delayed self-feedback loop to stabilize hidden state transitions.
result ERNN achieves state-of-the-art results on benchmark datasets.
This work improves LSTM and GRU training stability and generalization.
problem Training instabilities in LSTMs and GRUs on long sequences.
method Developed a mean field theory to optimize initialization hyperparameters.
result Eliminates or reduces training instabilities and improves generalization.
Noise stability improves understanding of Transformer models.
problem Lack of robustness metrics for real-valued domains and junta-like input dependence in modern LLMs.
method Proposed noise stability as a new metric and developed a practical regularization method.
result Noise stability regularization method accelerates training by 35-75%.
CGNNs use wavelets for continuous function generation in infinite-dimensional spaces.
problem Generating continuous functions in infinite-dimensional spaces for applications like inverse problems.
method Inspired by DCGAN, CGNNs use wavelet multiresolution analysis with convolutional and nonlinear layers.
result CGNNs can be injective under certain conditions on filters and nonlinearity, leading to Lipschitz stability estimates.
Proposes a new RNN structure to improve expressivity without sacrificing stability.
problem Exploding and vanishing gradient problems in RNNs and reduced expressivity.
method Introduces a non-normal RNN structure using Schur decomposition and splitting.
result Enhances expressivity while maintaining stability and training speed.
Paper analyzes GCNN sensitivity to probabilistic graph perturbations.
problem Investigating how GCNNs handle probabilistic graph errors.
method Establishes error bounds and linear relationships between GSO perturbations and GCNN outputs.
result GCNNs maintain stability under graph edge perturbations if GSO errors are bounded.
A new geometry for comparing signals, overcoming traditional limitations.
problem Comparing and interpolating discontinuous and signed signals.
method Investigation of Riemannian geometry on signal space, introducing a metric that measures both horizontal and vertical deformations.
result Characterization of metric properties and establishment of geodesic regularity and stability.
Framework for designing nonlinearities in neural networks with slope constraints.
problem Designing nonlinearities with specific properties for signal processing.
method Variational framework with regularization for slope constraints and optimization of adaptive splines.
result Adaptive nonuniform linear splines achieve global optimum in constrained optimization.
GNNs are stable to small topology changes and distinguish high frequencies.
problem Impact of topology changes on GNN outputs.
method Permutation equivariance, integral Lipschitz filters, frequency mixing.
result GNNs are stable to small topology changes and distinguish high frequencies.
Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which i…
We analyze why some models resist unlearning using linear stability theory.
problem Understanding and predicting when machine learning models resist unlearning.
method Linear stability theory applied to machine learning models, focusing on data coherence and optimization dynamics.
result Data coherence and signal-to-noise ratio (SNR) influence unlearning resistance; lower SNR makes unlearning easier.
Study finds macroeconomic indicators predict health workforce and infrastructure measures.
problem Evaluating the predictive value of macroeconomic indicators for public health targets.
method Examined multiple forecasting approaches including neural networks, generalized additive models, random forests, and time series models with exogenous indicators.
result Macroeconomic indicators provide consistent and reproducible predictive signals for health workforce and infrastructure measures, but less so for other targets.
Neural network tackles continual learning with neuromodulation and local error signals.
problem Catastrophic forgetting in continuous learning.
method Biologically-inspired neural architecture with local learning and neuromodulation, combined with transfer metalearning.
result Superior performance in continual learning tasks compared to other approaches.
Designs CNNs for better image reconstruction.
problem Image reconstruction from limited data.
method Parseval convolution operators and chaining of elementary modules.
result CNN-based algorithm yields better results than sparsity-based methods.
Stabilizes training of DNN for speech enhancement using PESQ scores.
problem Stability issues in training DNNs using non-differentiable OSQA scores.
method Approximate OSQA scores with a differentiable auxiliary DNN and stabilize training with reinforcement learning techniques.
result Stable training of DNN to achieve state-of-the-art PESQ scores and better sound quality.
RINS-T solves time series inverse problems robustly without pretraining.
problem Recovering original signals from corrupted time series data.
method Implicit neural solvers with robust optimization techniques.
result RINS-T achieves high recovery performance without pretraining.
Generalizes PCA and ICA for continuous-time signals using neural networks.
problem Low-rank decomposition of continuous-time vector-valued signals.
method Implicit neural network framework to learn numerical approximations of PCA and ICA.
result Unified approach to PCA and ICA in continuous domain, enforcing decorrelation and independence.
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
problem Bayesian neural networks struggle with choosing appropriate priors, leading to over-shrinkage or poor predictive performance.
method Proposes R2D2-Net with an R^2-induced Dirichlet Decomposition prior and variational Gibbs inference algorithm.
result R2D2-Net effectively shrinks irrelevant coefficients while preventing key features from over-shrinkage.
Improved language identification accuracy through signal combination methods.
problem Enhancing speech recognition accuracy across multiple languages.
method Combining low-level acoustic signals with language-specific recognizer signals using lattice-based ensemble models and deep neural networks.
result Deep neural network model outperforms lattice-based ensemble model, reducing error rate from 5.5% to 4.3%.
Neural network detects spectrum signals under uncertain parameters.
problem Spectrum sensing under uncertain primary user signal parameters.
method Trained neural network to detect modulated signals robustly.
result Neural network gains robustness under carrier frequency, phase, and symbol time offsets.
GNN improves financial risk detection in dynamic networks.
problem Complex, changing financial networks make traditional risk identification methods ineffective.
method Graph Neural Networks (GNN) for embedded representation learning of financial data.
result GNN enhances the detection of hidden risks and abnormal behaviors in financial networks.
New method uses DNN for genetic variant identification, controlling randomness and improving interpretability.
problem Challenges in interpreting deep neural networks for genetic variant identification.
method Interpretable neural network model with controlled variable selection using ensembling, knockoffs, and de-randomization.
result The proposed method leads to more discoveries compared to conventional methods.
New methods implement manifold scattering transform for high-dimensional point cloud data.
problem Classifying data on complex, non-linear manifolds.
method Adapting diffusion maps theory for numerical implementation.
result Effective for signal and manifold classification tasks.
DyEnsemble improves BCI accuracy by adapting to nonstationary neural signals.
problem Nonstationary neural signals in BCI cause decoding errors.
method Dynamic ensemble modeling that learns and combines diverse models online.
result DyEnsemble outperforms Kalman filters, especially with noisy signals.
Retraining stabilizes model influence on data.
problem Performativity in predictive models leads to feedback loops.
method Developed the stable signal principle to address retraining dynamics.
result Repeated risk minimization converges geometrically to stable signal direction.
Neural signals are characterized by rich temporal and spatiotemporal dynamics that reflect the organization of cortical networks. Theoretical research has shown how neural networks can operate at different dynamic ranges that correspond to specific types of information processing. Here we present a data analysis framew…