Deep nets trained with MSE loss exhibit Neural Collapse, collapsing features and classifiers to class means.
problem Understanding Neural Collapse in MSE-trained deep nets.
method Developed a new MSE loss decomposition and introduced the central path concept.
result Exact dynamics of Neural Collapse along the central path can be predicted.
In this manuscript we propose two objective terms for neural image compression: a compression objective and a cycle loss. These terms are applied on the encoder output of an autoencoder and are used in combination with reconstruction losses. The compression objective encourages sparsity and low entropy in the activatio…
This work justifies neural collapse under MSE loss and analyzes the optimization landscape.
problem Understanding neural collapse in deep neural networks under MSE loss.
method Global landscape analysis of vanilla nonconvex MSE loss.
result The only global minimizers are neural collapse solutions.
We derive the mapping between two of the most pervasive utility functions, the mean square error (MSE) and the concordance correlation coefficient (CCC, ρc). Despite its drawbacks, MSE is one of the most popular performance metrics (and a loss function); along with lately ρc in many of the sequence prediction…
This paper solves hedging in incomplete markets using neural networks.
problem Hedging in incomplete markets with risk factor, illiquidity, and discrete transaction dates.
method Proposes a jump-diffusion model and uses RNN, LSTM, and Mogrifier-LSTM neural networks for hedging strategies.
result Mogrifier-LSTM is the fastest and most effective model for hedging.
Enhances ENet's prediction accuracy while maintaining uncertainty estimation.
problem Gradient shrinkage problem in ENet's loss function.
method Proposes a multi-task learning framework with a modified MSE loss function.
result Improves ENet's prediction accuracy without losing uncertainty estimation.
With the recent advancement in the deep learning technologies such as CNNs and GANs, there is significant improvement in the quality of the images reconstructed by deep learning based super-resolution (SR) techniques. In this work, we propose a robust loss function based on the preservation of edges obtained by the Can…
This work introduces a new loss function to improve the efficiency of optimization-based PDE solvers.
problem Optimization-based PDE solvers converge slowly and are inefficient compared to classical iterative solvers.
method Proposes a novel Stabilized Gradient Residual (SGR) loss function to modulate the condition number.
result The SGR loss achieves orders-of-magnitude faster convergence than the MSE loss in both ODIL and PINNs frameworks.
The paper explores MAE as a loss function for DNN vector-to-vector regression, proving its advantages over MSE.
problem Improving loss function for deep neural network based vector-to-vector regression.
method Presenting performance bounds and new properties of MAE, deriving generalized upper bounds, and interpreting MAE as a Laplacian distribution.
result MAE is a more suitable loss function than MSE for DNN based vector-to-vector regression, especially when errors follow a Laplacian distribution.
New loss function improves accuracy of MRI parameter estimation.
problem Systematic errors in parameter estimates at low SNR.
method Developed and implemented negative log Rician likelihood (NLR) loss.
result NLR loss shows higher accuracy in parameter estimation than MSE loss at low SNR.
Replacing MSE with f-divergence in diffusion models improves robustness under data contamination.
problem Improving robustness of diffusion models under data contamination.
method Replacing MSE with f-divergence in diffusion models.
result Empirical improvement in performance under data contamination.
Improved speech enhancement using diffusion models with MSE loss.
problem Efficient incorporation of noisy speech in generative speech enhancement.
method Augmented diffusion-based generative model with a MSE loss for enhanced speech.
result Proposed method improves speech enhancement performance compared to original diffusion model.
We study the problem of sampling a bandlimited graph signal in the presence of noise, where the objective is to select a node subset of prescribed cardinality that minimizes the signal reconstruction mean squared error (MSE). To that end, we formulate the task at hand as the minimization of MSE subject to binary constr…
RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.
problem Improving performance of deep residual neural networks (RFNNs) while preserving convex optimization benefits.
method Random Feature Representation Boosting (RFRBoost) using boosting theory and random features at each layer.
result RFRBoost significantly outperforms RFNNs and end-to-end trained MLP ResNets in small- to medium-scale tabular datasets.
A new KF handles outliers without MSE loss.
problem Outliers degrade Kalman filter performance.
method NUV priors, EM and AM for variance estimation.
result Outlier-insensitive KF outperforms existing methods.
GMADL loss function improves model performance and reduces transaction costs.
problem Overfitting and high transaction costs in high-frequency algorithmic trading models.
method Introduces GMADL loss function for better optimization and feature selection.
result GMADL produces superior results and reduces transaction costs compared to standard loss functions.
New method improves model calibration by adjusting confidence based on prediction correctness.
problem Improving model confidence alignment with true class probabilities.
method Post-hoc calibration objective using transformed samples for training.
result Competitive calibration performance on in-distribution and out-of-distribution test sets.
Paper proposes a new autoencoder metric for balanced learning in imbalanced tabular datasets.
problem Challenges of imbalanced self-supervised learning in tabular data.
method Developed a Multi-Supervised Balanced MSE metric to balance learning.
result The new metric outperforms standard MSE in imbalanced datasets.
This paper examines how different loss functions affect neural network features and performance.
problem Investigating which loss function is best for deep neural networks.
method Examining last-layer features of deep networks and drawing inspiration from the Neural Collapse phenomenon.
result All relevant loss functions (CE, LS, FL, MSE) produce equivalent features and similar performance.
This work addresses unstable MeanFlow training by optimizing a coefficient in the loss function.
problem Unstable training of MeanFlow models with non-decreasing loss and unbounded gradient variance.
method Established a theory attributing the instability to misuse of the conditional velocity field, derived the optimal coefficient, and showed practical realizations.
result Optimal coefficient yields up to 54% improvement in sample quality and monotone FID trend.
New loss function reduces outage probability in ML-assisted resource allocation.
problem Minimizing outage probability in ML-assisted resource allocation systems.
method Developed a novel loss function and trained an ML model to address the outage probability challenge.
result Exact and asymptotic expressions for the system's outage probability were established.
Study shows MSE with sigmoid can match SCE in classification tasks, especially with noisy data.
problem Inconsistent errors in neural network classification tasks.
method Introduced Output Reset algorithm to use MSE with sigmoid activation.
result MSE with sigmoid activation achieves comparable accuracy and convergence rates to Softmax Cross-Entropy, especially in noisy data scenarios.
Improves generative models by adding jump-diffusion noise.
problem Limited performance of diffusion models in generating samples from unknown distributions.
method Generalizes diffusion processes to include jump-diffusion noise, deriving closed-form generalized score functions.
result Jump-diffusion models outperform Gaussian models in specific parameter regimes.
RG-TTA adapts neural forecasters to streaming time series shifts by modulating adaptation intensity.
problem Adapting neural forecasters to distribution shifts in streaming time series data.
method RG-TTA uses a meta-controller that continuously modulates adaptation intensity based on distributional similarity.
result RG-TTA achieves the lowest MSE in 156 of 224 seed-averaged experiments, reducing MSE by 5.7% vs TTA.
This article presents valuation of Treasury Bonds (T-Bonds) on Macedonian Stock Exchange (MSE) and empirical test of duration, modified duration and convexity of the T-bonds at MSE in order to determine sensitivity of bonds prices on interest rate changes. The main goal of this study is to determine how standard valuat…
Denoising autoencoders (DAEs) are powerful deep learning models used for feature extraction, data generation and network pre-training. DAEs consist of an encoder and decoder which may be trained simultaneously to minimise a loss (function) between an input and the reconstruction of a corrupted version of the input. The…
MeanFlow training is unstable due to misusing conditional velocity, leading to variance issues.
problem Unstable training of MeanFlow due to variance problems.
method Theoretical analysis and derivation of optimal coefficient in closed form.
result The optimal coefficient in MeanFlow training minimizes variance but not necessarily quality.
Develops efficient algorithm for distilling supervised and offline RL datasets.
problem Creating synthetic datasets for supervised and offline RL models to match training data performance.
method An efficient algorithm based on matching losses without model training, using randomly sampled regressors.
result Synthetic datasets derived with ildeO(d2) sampled regressors match MSE loss of bounded linear models on training data. Gaussian surrogates improve Poisson imaging performance at low doses.
problem Improving Poisson imaging performance at low doses.
method Analysis of Poisson and Gaussian surrogate reconstruction objectives under Poisson noise.
result Gaussian surrogates can achieve MSE comparable to Poisson MAP at low doses.
Causal machine-learning is about predicting the net-effect (true-lift) of treatments. Given the data of a treatment group and a control group, it is similar to a standard supervised-learning problem. Unfortunately, there is no similarly well-defined loss function due to the lack of point-wise true values in the data. M…
New theory shows large learning rates prevent overfitting in neural networks.
problem Generalization of two-layer ReLU neural networks in noisy regression problems.
method Gradient descent with constant learning rate converges to stable minima.
result Gradient descent with large learning rates finds smooth, sparse fits.
Uniform bounds for neural network convergence without strong convexity assumptions.
problem Understanding the convergence of neural networks in the feature-learning regime.
method Establishing uniform-in-time weak propagation-of-chaos via mean-field deterministic Wasserstein-gradient-flow dynamics.
result Uniform bounds on the difference between infinite-width and finite-width neural network outputs, showing that fewer neurons can achieve a desired loss.
Deep linear networks exhibit collapsing features and classifiers across datasets.
problem Understanding the collapse of features and classifiers in deep linear networks.
method Theoretical and empirical analysis of deep linear networks with MSE and CE losses.
result Deep linear networks exhibit NC properties, collapsing features and classifiers to orthogonal vectors.
Continuous monitoring of cardiac health under free living condition is crucial to provide effective care for patients undergoing post operative recovery and individuals with high cardiac risk like the elderly. Capacitive Electrocardiogram (cECG) is one such technology which allows comfortable and long term monitoring t…
Data symmetries in neural networks can generate conserved quantities.
problem Conservation laws in neural networks
method Using tensorizable networks
result Data augmentation can induce conserved quantities
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
Study evaluates uncertainty in BP estimation from PPG signals under domain shift.
problem Uncertainty quantification in healthcare, especially for cuffless BP estimation.
method Compared deep ensembles, Monte Carlo dropout, and various recalibration techniques.
result Deep ensembles provide stronger robustness under domain shift.
Enhances Transformers for better risk assessment in finance.
problem Transformer models lack sensitivity to extreme financial losses.
method Integrates Loss-at-Risk function with Value at Risk (VaR) and Conditional Value at Risk (CVaR).
result Improves risk prediction and management in financial datasets.
Deep learning detects microsleep episodes in EEG data.
problem Automatic detection of microsleep episodes (MSEs) in EEG data.
method Convolutional neural networks (CNNs) and LSTM networks were implemented to analyze MWT data.
result Deep learning algorithms showed good performance close to human experts in detecting MSEs.
Paper proposes HIDAM model to improve MSE default risk assessment using heterogeneous information networks.
problem Default risk assessment for MSEs due to lack of credit information and diverse financial activities.
method HIDAM model incorporating heterogeneous information networks with multi-typed nodes and links, extracting interactive information through meta-paths, and using a hierarchical attention mechanism.
result HIDAM model outperforms state-of-the-art competitors on real-world banking data.
This paper addresses the problem of time series forecasting for non-stationary signals and multiple future steps prediction. To handle this challenging task, we introduce DILATE (DIstortion Loss including shApe and TimE), a new objective function for training deep neural networks. DILATE aims at accurately predicting s…
Noise in imputed values corrects biases in machine learning models.
problem Systematic biases in imputed values affect downstream analyses.
method Introducing noise to imputed values to correct biases.
result Noise-corrected imputation methods produce unbiased estimates.
Paper uses DFL to optimize portfolio risk and outperforms conventional methods.
problem Optimizing portfolio risk and return under uncertainty.
method Decision-focused learning (DFL) to derive global minimum variance portfolio (GMVP).
result DFL-based methods consistently deliver superior decision performance in portfolio optimization.
Proposes new loss functions for better handling bimodal predictive uncertainty.
problem Bimodal predictive uncertainty in machine learning models.
method Family of distribution-aware loss functions integrating normalized RMSE with Wasserstein and Cramér distances.
result Proposed loss functions reduce predictive uncertainty estimation error by 45% on complex bimodal datasets.
Supervised learning based on a deep neural network recently has achieved substantial improvement on speech enhancement. Denoising networks learn mapping from noisy speech to clean one directly, or to a spectrum mask which is the ratio between clean and noisy spectra. In either case, the network is optimized by minimizi…
CAEL-MIPS learns embeddings to improve MIPS for better OPE in contextual bandits.
problem High variance in IPS weighting for OPE in large action spaces.
method Context-Action Embedding Learning (CAEL) for MIPS to minimize MSE.
result CAEL-MIPS outperforms baselines in MSE for OPE in contextual bandits.
The paper explains why estimating a history-dependent policy can reduce MSE in reinforcement learning.
problem Understanding why history-dependent policies can improve MSE in off-policy evaluation.
method The paper derives a bias-variance decomposition of MSE for various OPE estimators, showing how history-dependent policies can decrease variance and increase bias.
result History-dependent policies can decrease the variance of importance sampling estimators, leading to lower MSE.
Weather forecasting is usually solved through numerical weather prediction (NWP), which can sometimes lead to unsatisfactory performance due to inappropriate setting of the initial states. In this paper, we design a data-driven method augmented by an effective information fusion mechanism to learn from historical data …