Loss-calibrated EP improves Bayesian decision-making by focusing on utility-sensitive posterior approximations.
problem Bayesian decision-making under asymmetric utility functions.
method Loss-calibrated expectation propagation (Loss-EP) that tilts the posterior towards higher utility decisions.
result Loss-EP can capture useful information for decision-making under asymmetric penalties.
We consider a dynamical model of distress propagation on complex networks, which we apply to the study of financial contagion in networks of banks connected to each other by direct exposures. The model that we consider is an extension of the DebtRank algorithm, recently introduced in the literature. The mechanics of di…
Theoretical framework for target propagation shows differences from backpropagation.
problem Understanding and improving target propagation for neural networks.
method Mathematical optimization analysis and novel reconstruction loss.
result A novel reconstruction loss improves feedback weight training and introduces architectural flexibility.
Proposes a semi-implicit back propagation method for neural networks.
problem Challenges in training neural networks, especially gradient vanishing and small step sizes.
method Proposes a semi-implicit back propagation method using error back propagation and proximal methods.
result The proposed method leads to better performance in terms of loss decreasing and training/validation accuracy compared to SGD and ProxBP.
New method predicts propagation losses at 169 MHz for smart metering.
problem Radio planning for 169 MHz smart metering networks.
method Support Vector Machine techniques for classification and regression.
result Good accuracy achieved at low computational cost and minimal measurement effort.
Neural network with loss ensemble improves text classification accuracy.
problem Improving text classification accuracy in noisy environments.
method Extended neural network with an ensemble loss function, weights tuned through gradient propagation.
result Improvement in classification accuracy and resilience against label noise.
TD learning with neural networks can lead to worse solutions than Monte-Carlo methods, especially in discontinuous value functions.
problem TD learning with neural networks can propagate approximation errors, leading to worse solutions than Monte-Carlo methods.
method Investigated the issue of approximation errors in areas of sharp discontinuities of the value function being further propagated by bootstrap updates.
result Empirical and analytical evidence shows that leakage propagation occurs in TD learning with function approximation, especially in sharp discontinuities.
A new approach estimates propagators for trading risky assets.
problem Estimating price impact kernel from static data for optimal trading.
method Nonparametric estimation of propagator using offline reinforcement learning.
result Pessimistic trading strategy optimises execution costs under uncertainty.
This article describes a multivariate polynomial regression method where the uncertainty of the input parameters are approximated with Gaussian distributions, derived from the central limit theorem for large weighted sums, directly from the training sample. The estimated uncertainties can be propagated into the optimal…
CoPhy-PGNN tackles competing PG losses in neural networks for solving eigenvalue problems.
problem Solving eigenvalue problems with competing physics-guided loss functions.
method Learning generalizable solutions using a novel approach to handle competing PG losses.
result Demonstrates the effectiveness of the approach in quantum mechanics and electromagnetic propagation.
Proposes Group Loss for deep metric learning to improve clustering and image retrieval.
problem Improving deep metric learning for better clustering and image retrieval.
method Group Loss based on label-propagation method enforcing embedding similarity across all samples of a group.
result Shows state-of-the-art results on clustering and image retrieval on several datasets.
This paper proposes an alternative to E2E training for deep networks, reducing memory footprint.
problem High GPUs memory footprint in end-to-end training of deep networks.
method Locally supervised learning with information propagation loss to avoid information collapse.
result The proposed method achieves competitive performance with less than 40% memory footprint compared to E2E training.
Two new algorithms improve Q* approximation in batch RL with linear error propagation.
problem Improving Q* approximation in batch reinforcement learning.
method Two novel algorithms that estimate Bellman error directly, without quadratic dependence.
result Linear-in-horizon error propagation for batch RL algorithms.
Improved WBP decoding with simple scaling and SNR adaptation.
problem Efficiently decoding weighted Tanner graphs with reduced complexity.
method Simple-scaling models with machine learning for edge weights, and parameter adapter networks.
result Simple scaling with few parameters can achieve near-maximum-likelihood performance.
Using particle system methodologies we study the propagation of financial distress in a network of firms facing credit risk. We investigate the phenomenon of a credit crisis and quantify the losses that a bank may suffer in a large credit portfolio. Applying a large deviation principle we compute the limiting distribut…
IBP simplifies robust training for large networks.
problem Training robust neural networks with scalable methods.
method Interval Bound Propagation (IBP) for robust training.
result IBP enables training large provably robust networks.
A new framework explains mixed models by propagating Shapley values.
problem Making complex models like neural networks and stacked models explainable for healthcare applications.
method DeepSHAP framework for layer-wise propagation of Shapley values.
result DeepSHAP enables attributions for mixed models and theoretically justifies attributions with respect to a background distribution.
Optimization-based pruning eliminates backpropagation for large language models.
problem Suboptimal pruning performance due to heuristic metrics.
method Optimization of Bernoulli distribution to learn pruning masks without backpropagation.
result Efficient pruning of large language models with improved performance.
Uniform bounds for neural network convergence without strong convexity assumptions.
problem Understanding the convergence of neural networks in the feature-learning regime.
method Establishing uniform-in-time weak propagation-of-chaos via mean-field deterministic Wasserstein-gradient-flow dynamics.
result Uniform bounds on the difference between infinite-width and finite-width neural network outputs, showing that fewer neurons can achieve a desired loss.
Proposes a method to quantify uncertainty in graph neural networks for node classification.
problem Uncertainty in graph neural networks for node classification.
method Bayesian uncertainty propagation (BUP) method embedding GNNs in a Bayesian framework.
result Demonstrates superior performance of the proposed method on benchmark datasets.
Back-propagation learns camera sensor design for color images.
problem Designing efficient color camera sensors for deep learning.
method Jointly learns sensor design and image reconstruction networks.
result Significant accuracy improvements over traditional Bayer pattern.
Network-based stress test assesses central counterparty resilience.
problem Quantifying resilience of central counterparties during financial distress.
method Network analysis of clearing members, simulating financial distress propagation.
result Default funds may not be adequate for systemic events, requiring conservative amounts.
New method reduces memory usage in deep HRNNs by replacing gradient backpropagation with local losses.
problem Memory constraints in training deep hierarchical RNNs.
method Replace gradient backpropagation with locally computable losses in deep HRNNs.
result Memory requirements reduced by a factor exponential in hierarchy depth.
New insights into neural network initialization and activation functions improve deep learning performance.
problem Inappropriate initialization and activation function selection can hinder deep neural network training.
method Theoretical analysis and quantitative results on weight initialization and activation functions.
result Random initialization at the edge of chaos improves information propagation in deep neural networks.
This work improves structured prediction by learning the balance between signal and random noise.
problem Structured prediction with random perturbations.
method Learning the variance of randomized structured predictors to balance signal and noise.
result Learning the balance improves structured prediction effectiveness.
Dropout schedules can be optimized to significantly reduce model test loss.
problem Improving model performance in neural networks.
method Developed a mean-field theory of dropout at the edge of chaos, proposing front-loaded dropout schedules.
result Front-loaded dropout schedules reduce test loss by 18-35% over constant dropout.
We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…
A new method quantizes LSTM gate parameters without performance loss.
problem Quantization loss in LSTM gate parameters without performance degradation.
method Lossy quantization of gate parameters during training, weight parameters adjust to offset quantization loss.
result F1 score decreased by only 0.7% on Named Entity Recognition dataset.
Study ruin probabilities in risk processes on stochastic networks.
problem Ruin probabilities in risk processes on stochastic networks.
method Classification of agents by types, Poisson process for loss propagation, explicit ruin probabilities for infinite network size.
result Explicit ruin probabilities for agents of any type in infinite network size.
New loss functions make deep nets robust to noisy labels.
problem Label noise in training data affects deep neural networks.
method Developed conditions for loss functions to be robust to label noise.
result Mean absolute value loss is inherently robust to label noise.
A simplified backpropagation method reduces model complexity and computational cost.
problem Complexity and computational inefficiency in deep learning models.
method Sparsification of gradient vectors and adaptive model simplification.
result Models can be simplified by updating only a small fraction of weights, improving accuracy.
Paper aims to ensure reliable detection of out-of-distribution data with certifiable worst-case guarantees.
problem Deep neural networks are overconfident with OOD inputs, posing safety risks.
method Enforces low confidence and bounds in an l∞-ball around OOD points using interval bound propagation (IBP). result Certifiable worst-case guarantees for OOD detection are possible without significant loss in accuracy.
PPN learns from weakly-labeled data to improve few-shot learning.
problem Few-shot learning with limited labeled data.
method Prototype Propagation Network (PPN) trained on few-shot tasks with coarse-label data.
result PPN significantly outperforms other methods on benchmarks.
Proposes IFCDA framework to improve cross-domain adaptation.
problem Negative transfer and difficulty in handling category-irrelevant losses in DA.
method Importance filtered mechanism to generate filtered soft labels, combined with graph-based label propagation.
result Significantly improves performance in both Closed-Set and Open-Set DA scenarios.
Unified framework for faster neural network training with less information loss.
problem Time-consuming backpropagation and loss of unpropagated gradient information.
method Unified sparse backpropagation framework and memorized sparse backpropagation algorithm.
result Convergence in probability with certain conditions and effective information loss mitigation.
XAI identifies key time steps for early crop classification.
problem Early crop classification with high accuracy and timeliness.
method Training a baseline model with LRP to identify important time steps.
result Identified a 21st April 2019 to 9th August 2019 timeframe with 0.75% accuracy loss.
New method trains neural networks with local error signals, outperforming global methods.
problem Training neural networks with global error signals.
method Layer-wise training with local error signals.
result Layer-wise training with local error signals can approach state-of-the-art performance.
OpticNet predicts IOL optical properties from biometric data, outperforming current methods.
problem Precise prediction of IOL optical properties for cataract surgery.
method Unsupervised, domain-specific, physically motivated optical refraction network.
result OpticNet outperforms current methods in predicting IOL optical properties.
The paper proposes new cross-correlators using Price's Theorem and piecewise-linear decomposition.
problem Optimal method for estimating cross-correlations using finite samples.
method General mathematical framework using Price's Theorem and piecewise-linear decomposition.
result Some cross-correlators based on Huber's loss functions, MP functions, and LSE functions have higher SNR.
Paper predicts GNSS phase scintillations with machine learning.
problem Predicting phase scintillations due to ionosphere disturbances.
method Proposes a novel machine learning architecture and loss function.
result Achieves state-of-the-art prediction of phase scintillations 1 hour in advance.
Study network equilibria in saturated systems, revealing how small shocks can trigger major losses.
problem Understanding how small shocks can lead to major losses in financial networks and games.
method Derived explicit expressions for network equilibria, proved conditions for their uniqueness, and analyzed discontinuities.
result Bifurcation phenomenon in network equilibria, showing sensitivity to small shocks.
A new method prunes activation gradients to speed up CNN training.
problem Challenges in accelerating CNN training using sparsity.
method Randomly prunes small activation gradients in back-propagation.
result Substantial speedups (up to 5.92x) with minimal accuracy loss.
Learning with noisy labels is one of the hottest problems in weakly-supervised learning. Based on memorization effects of deep neural networks, training on small-loss instances becomes very promising for handling noisy labels. This fosters the state-of-the-art approach "Co-teaching" that cross-trains two deep neural ne…
Proposes variational Gaussian approximations for solving the Kushner equation.
problem Solving the Kushner equation for state estimation with observations.
method Tractable variational Gaussian approximations of proximal losses based on Wasserstein and Fisher metrics.
result The proposed method leads to a Gaussian flow consistent with Kalman-Bucy and Riccati flows.
Study on learning to defer to multiple experts with consistent surrogates and confidence calibration.
problem Addressing the open problems of consistent surrogates, confidence calibration, and ensembling of experts.
method Derive two consistent surrogates (softmax and OvA) and propose a conformal inference technique for choosing experts.
result The OvA-based loss does not cause mis-calibration propagation, while the softmax-based loss does.
Proves well-posedness for Einstein equations with totally geodesic timelike boundary condition.
problem Initial boundary value problem for Einstein equations with specific geometric boundary condition.
method ADM system, parallelly propagated orthonormal frame, modified evolution equations, hyperbolic systems, constraints propagation.
result First well-posedness result for Einstein equations with totally geodesic timelike boundary condition.
This research develops a new model for cyber risk and insurance pricing.
problem Accurate calculation of aggregate losses in cyber insurance pricing.
method A path-based k-generation risk contagion model in a tree-shaped network structure.
result Explicit expressions for mean and variance of local loss on a single path.
A new method prevents forgetting during knowledge transfer.
problem Catastrophic forgetting in transfer learning.
method Transfer without Forgetting (TwF) using a fixed pretrained network.
result TwF outperforms other CL methods by 4.81% in Class-Incremental accuracy.