A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper proposes efficient weight updates for edge nodes with minimal communication.
We show that deep networks can be trained using Hebbian updates yielding similar performance to ordinary back-propagation on challenging image datasets. To overcome the unrealistic symmetry in connections between layers, implicit in back-propagation, the feedback weights are separate from the feedforward weights. The f…
In this paper, we present a novel approach for fine-tuning a decoder-side neural network in the context of image compression, such that the weight-updates are better compressible. At encoder side, we fine-tune a pre-trained artifact removal network on target data by using a compression objective applied on the weight-u…
RSO uses random weight perturbations to train deep networks without gradients.
Paper proves multiplicative weight updates can train neural networks without learning rate tuning.
We describe -MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…
Online learning makes sequence of decisions with partial data arrival where next movement of data is unknown. In this paper, we have presented a new technique as multiple times weight updating that update the weight iteratively forsame instance. The proposed technique analyzed with popular state-of-art algorithms from …
Bio-inspired neural networks use predictive coding for efficient weight updates.
Paper proposes a new activation function to reduce overfitting and large weight update issues.
This paper optimizes deep learning training by efficiently sharding weight updates across replicas.
Optimization algorithms with momentum, e.g., (ADAM), have been widely used for building deep learning models due to the faster convergence rates compared with stochastic gradient descent (SGD). Momentum helps accelerate SGD in the relevant directions in parameter updating, which can minify the oscillations of parameter…
Fewer data weight updates lead to faster convergence in machine learning models.
New algorithm for online optimization over symmetric cones, unifying previous methods.
Spectral Adaptive Conformal Prediction for Structured Non-Exchangeable Data
Efficiently optimizes boolean functions using multilinear polynomials and exponential weight updates.
Sparse random networks reduce communication in federated learning.
This paper studies recursive ensembles driven by Fibonacci updates, improving learning dynamics.
This paper considers the actor-critic contextual bandit for the mobile health (mHealth) intervention. The state-of-the-art decision-making methods in mHealth generally assume that the noise in the dynamic system follows the Gaussian distribution. Those methods use the least-square-based algorithm to estimate the expect…
Importance sampling (IS) is a common reweighting strategy for off-policy prediction in reinforcement learning. While it is consistent and unbiased, it can result in high variance updates to the weights for the value function. In this work, we explore a resampling strategy as an alternative to reweighting. We propose Im…
Improves inference-time alignment for diffusion models without updating weights.
Efficiently updates beliefs with virtual observations.
In importance sampling (IS)-based reinforcement learning algorithms such as Proximal Policy Optimization (PPO), IS weights are typically clipped to avoid large variance in learning. However, policy update from clipped statistics induces large bias in tasks with high action dimensions, and bias from clipping makes it di…
A RL approach dynamically assigns and updates weights of ensemble models for better time series forecasting.
We consider a setting where an agent's uncertainty is represented by a set of probability measures, rather than a single measure. Measure-bymeasure updating of such a set of measures upon acquiring new information is well-known to suffer from problems; agents are not always able to learn appropriately. To deal with the…
A new accelerated method with simpler momentum update rules.
We present a confidence-based single-layer feed-forward learning algorithm SPIRAL (Spike Regularized Adaptive Learning) relying on an encoding of activation spikes. We adaptively update a weight vector relying on confidence estimates and activation offsets relative to previous activity. We regularize updates proportion…
Bayesian neural networks learn weights with closed-form updates.
The decremented learning algorithms are required in machine learning, to prune redundant nodes and remove obsolete inline training samples. In this paper, an efficient decremented learning algorithm to prune redundant nodes is deduced from the incremental learning algorithm 1 proposed in [9] for added nodes, and two de…
GFM models neural network training as a dynamical system to forecast final weights.
A novel Bayesian computation method using importance weighting improves numerical stability and performance.
Pion optimizes LLMs by preserving weight matrix singular values.
Current deep neural networks (DNNs) can easily overfit to biased training data with corrupted labels or class imbalance. Sample re-weighting strategy is commonly used to alleviate this issue by designing a weighting function mapping from training loss to sample weight, and then iterating between weight recalculating an…
EP learns like BPTT but with local weight updates.
New algorithm uses PSO to optimize DNN training parameters in distributed systems.
Study on learning rates in neural networks of varying depth.
Gradient-based methods introduce noise that penalizes models sensitive to weight perturbations.
Enhances linear regression with Kalman filter for loss minimization.
TSC improves causal effect estimation in panel data.
Non-negative matrix factorization is a popular tool for decomposing data into feature and weight matrices under non-negativity constraints. It enjoys practical success but is poorly understood theoretically. This paper proposes an algorithm that alternates between decoding the weights and updating the features, and sho…
In this paper we revisit the weighted likelihood bootstrap, a method that generates samples from an approximate Bayesian posterior of a parametric model. We show that the same method can be derived, without approximation, under a Bayesian nonparametric model with the parameter of interest defined as minimising an expec…
In-Place TTT enhances LLMs with dynamic parameter updates at inference time.
We introduce a minorization-maximization approach to optimizing common measures of discovery significance in high energy physics. The approach alternates between solving a weighted binary classification problem and updating class weights in a simple, closed-form manner. Moreover, an argument based on convex duality sho…
Background: Deep learning models are typically trained using stochastic gradient descent or one of its variants. These methods update the weights using their gradient, estimated from a small fraction of the training data. It has been observed that when using large batch sizes there is a persistent degradation in genera…
To recover a sparse signal from an underdetermined system, we often solve a constrained L1-norm minimization problem. In many cases, the signal sparsity and the recovery performance can be further improved by replacing the L1 norm with a "weighted" L1 norm. Without any prior information about nonzero elements of the si…
Study reveals dynamics of neural networks with normalization, weight decay, and SGD.
New algorithm provably converges to second-order stationary points in NMF.
New sketches for weighted sampling without replacement improve accuracy and efficiency.
Traditionally, deep learning algorithms update the network weights whereas the network architecture is chosen manually, using a process of trial and error. In this work, we propose two novel approaches that automatically update the network structure while also learning its weights. The novelty of our approach lies in o…