Bias-free CNNs outperform biased ones in blind image denoising.
problem Deep learning methods often require bias terms, leading to overfitting and poor performance outside training noise levels.
method Developed a bias-free convolutional neural network architecture by removing additive constant terms in every layer, including batch normalization.
result Bias-free CNNs generalize robustly across noise levels, preserving state-of-the-art performance within the training range.
We propose a version of least-mean-square (LMS) algorithm for sparse system identification. Our algorithm called online linearized Bregman iteration (OLBI) is derived from minimizing the cumulative prediction error squared along with an l1-l2 norm regularizer. By systematically treating the non-differentiable regulariz…
The paper analyzes and mitigates biases in scalable Gaussian Process methods.
problem Modeling biases in scalable Gaussian Process methods.
method Randomized truncation estimators to eliminate bias in exchange for increased variance.
result Randomized truncation estimators meaningfully outperform biased counterparts with minimal additional computation.
In this paper, we made an extension to the convergence analysis of the dynamics of two-layered bias-free networks with one ReLU output. We took into consideration two popular regularization terms: the ℓ1 and ℓ2 norm of the parameter vector w, and added it to the square loss function with coefficient $λ/…
We consider reinforcement learning in input-driven environments, where an exogenous, stochastic input process affects the dynamics of the system. Input processes arise in many applications, including queuing systems, robotics control with disturbances, and object tracking. Since the state dynamics and rewards depend on…
Work establishes conditions for bias-free policy optimization.
problem Improving policy optimization methods without introducing bias.
method Established conditions for parametric critic without bias.
result Identified bias in current policy optimization algorithms.
Generalizing empirical findings to new environments, settings, or populations is essential in most scientific explorations. This article treats a particular problem of generalizability, called "transportability", defined as a license to transfer information learned in experimental studies to a different population, on …
Bias is essential for machine learning success, quantifiable and conserved.
problem The necessity and quantification of bias in machine learning success.
method Quantifying bias relative to possible datasets and demonstrating its role in increasing success probability.
result Bias is a conserved quantity and essential for favorably biasing towards a fixed target.
Policy gradient methods have enjoyed great success in deep reinforcement learning but suffer from high variance of gradient estimates. The high variance problem is particularly exasperated in problems with long horizons or high-dimensional action spaces. To mitigate this issue, we derive a bias-free action-dependent ba…
New method stabilizes deep neural networks by setting Lyapunov exponent to zero.
problem Stability issues in deep neural networks with low width.
method Lyapunov initialization method to set Lyapunov exponent to zero.
result Lyapunov exponent governs stability of deep networks; standard methods fail for low width.
AdvReg improves VQA models but introduces instability and bias issues.
problem VQA models over-rely on linguistic biases, ignoring visual context.
method Adversarial regularization to encourage bias-free question representations.
result AdvReg yields side-effects like unstable gradients and reduced performance on in-domain examples.
Neural networks and linear systems linked, revealing training loss and kernel limitations.
problem Exploring the training loss and limitations of neural networks and their kernels.
method Drawing connections between neural networks and under-determined linear systems, providing lower bounds, and analyzing gradient descent.
result Zero training loss achievable for neural networks under certain conditions, but not for ReLU kernels.
Deep linear networks exhibit collapsing features and classifiers across datasets.
problem Understanding the collapse of features and classifiers in deep linear networks.
method Theoretical and empirical analysis of deep linear networks with MSE and CE losses.
result Deep linear networks exhibit NC properties, collapsing features and classifiers to orthogonal vectors.
The paper introduces a method to achieve fairness in machine learning models using graph models.
problem Theoretical properties and intuition behind fairness in machine learning models are poorly understood.
method Sheaf Diffusion framework to model fairness in a bias-free space.
result The proposed method achieves fair solutions and handles different fairness metrics.
DIGing-SGLD improves SGLD for scalable Bayesian learning in dynamic networks.
problem Scalable Bayesian learning in multi-agent systems with time-varying networks.
method Integrates Langevin sampling with gradient-tracking for decentralized learning over time-varying networks.
result Achieves geometric convergence to the target distribution with finite-time guarantees.
New method smooths integrands for efficient option pricing.
problem Improving numerical performance of option pricing methods.
method Combining hierarchical adaptive sparse grids, quasi-Monte Carlo, and numerical smoothing.
result Improved efficiency of ASGQ and QMC methods for high-dimensional problems.
Study spectral density of neural networks using resolvent method.
problem Investigate spectral density of neural networks with random feature matrices.
method Use resolvent method from random matrix theory, cumulant expansion.
result Impossible to preserve singular value distribution with additive bias.
Study characterizes harmful low-fidelity data sources for surrogate models.
problem Identifying which low-fidelity data sources to use in constructing surrogate models.
method Employed benchmark filtering techniques to assess harmful sources using limited data.
result Provided guidelines for using low-fidelity sources in an industrial setting.
A new method called MCLMC avoids dissipation in sampling from canonical distributions.
problem Sampling from canonical distributions without dissipation.
method Microcanonical Langevin Monte Carlo (MCLMC) as a dissipation-free system of SDE.
result MCLMC converges faster than HMC for lattice φ^4 models.
In this paper, we recover sparse signals from their noisy linear measurements by solving nonlinear differential inclusions, which is based on the notion of inverse scale space (ISS) developed in applied mathematics. Our goal here is to bring this idea to address a challenging problem in statistics, \emph{i.e.} finding …