A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New model predicts video sequences with latent dynamics.
Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-du…
Nonnegative matrix factorization (NMF) has attracted much attention in the last decade as a dimension reduction method in many applications. Due to the explosion in the size of data, naturally the samples are collected and stored distributively in local computational nodes. Thus, there is a growing need to develop algo…
GT-DDP optimizer trains residual networks using game theory.
Paper proves Jeffrey's update rule minimizes relative entropy.
Extended LSTM improves language modeling performance.
A new update rule for deep reinforcement learning reduces learning variance and variance in reference signals.
A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this involves minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations us…
Unified learning-rate scale for CNNs and ResNets, avoiding depth imbalance.
New learning rule simplifies Bayesian updates for deep learning.
MTL2L learns to adapt optimisation rules for unseen data.
New decision-theoretic characterization separates belief and decision posteriors.
The study learns neural update rules by remembering past experiences.
Adversarial training is a technique for training robust machine learning models. To encourage robustness, it iteratively computes adversarial examples for the model, and then re-trains on these examples via some update rule. This work analyzes the performance of adversarial training on linearly separable data, and prov…
Belief propagation and its variants are popular methods for approximate inference, but their running time and even their convergence depend greatly on the schedule used to send the messages. Recently, dynamic update schedules have been shown to converge much faster on hard networks than static schedules, namely the res…
Study on incentivizing truthfulness in federated learning with heterogeneous data.
The paper introduces a diagnostic method to detect grokking transitions in models before test accuracy improves.
Stochastic gradient descent (SGD) has been the dominant optimization method for training deep neural networks due to its many desirable properties. One of the more remarkable and least understood quality of SGD is that it generalizes relatively well on unseen data even when the neural network has millions of parameters…
An online decision-making algorithm using stochastic gradient descent for big data.
Here, we study different update rules in stochastic gradient descent (SGD) for online forecasting problems. The selection of the learning rate parameter is critical in SGD. However, it may not be feasible to tune this parameter in online learning. Therefore, it is necessary to have an update rule that is not sensitive …
Nonnegative matrix factorization (NMF), a dimensionality reduction and factor analysis method, is a special case in which factor matrices have low-rank nonnegative constraints. Considering the stochastic learning in NMF, we specifically address the multiplicative update (MU) rule, which is the most popular, but which h…
Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works have approached this issue either by attempting to train a neural network that directly produces updates or by attempting to learn better i…
A new model improves CT image quality from low-dose scans.
Recent years have witnessed the rapid development of block coordinate update (BCU) methods, which are particularly suitable for problems involving large-sized data and/or variables. In optimization, BCU first appears as the coordinate descent method that works well for smooth problems or those with separable nonsmooth …
The growth of data, the need for scalability and the complexity of models used in modern machine learning calls for distributed implementations. Yet, as of today, distributed machine learning frameworks have largely ignored the possibility of arbitrary (i.e., Byzantine) failures. In this paper, we study the robustness …
Develops local population-risk certificates for model updates
RSI uses Bayesian inference to monitor compliance in rule-governed domains.
Non-affine aggregation rules cannot preserve monotonicity in convex learning.
A new method improves few-shot image classification by updating top layers.
TRA detects causal direction from bivariate data using geometric shapes.
New priors can update posteriors without re-estimating likelihoods.
Improved greedy 2-coordinate updates for optimization problems with constraints.
Test-time training adapts a pretrained model to each prompt via parameter updates, improving accuracy under pretraining-to-test distribution shifts.
Paper proposes a math-inspired L2O model for better generalization.
In this paper we provide a flexible framework allowing for a unified study of time consistency of risk measures and performance measures (also known as acceptability indices). The proposed framework not only integrates existing forms of time consistency, but also provides a comprehensive toolbox for analysis and synthe…
In this paper, two Q-learning (QL) methods are proposed and their convergence theories are established for addressing the model-free optimal control problem of general nonlinear continuous-time systems. By introducing the Q-function for continuous-time systems, policy iteration based QL (PIQL) and value iteration based…
New diagnostic method detects misspecified models in inverse PDE problems.
This research improves asset life prediction by integrating deep learning with mixture distributions.
FetchSGD reduces communication in federated learning with sketching.
In this paper, several modifications are introduced to the functional approximation method iterLap to reduce the approximation error, including stopping rule adjustment, proposal of new residual function, starting point selection for numerical optimisation, scaling of Hessian matrix. Illustrative examples are also prov…
Normalization layers are a staple in state-of-the-art deep neural network architectures. They are widely believed to stabilize training, enable higher learning rate, accelerate convergence and improve generalization, though the reason for their effectiveness is still an active research topic. In this work, we challenge…
Bayesian neural networks update beliefs with soft evidence, improving accuracy and calibration.
Context-awareness in smart mobile applications is a growing area of study, because of it's intelligence in the applications. In order to build context-aware intelligent applications, mining contextual behavioral rules of individual smartphone users utilizing their phone log data is the key. However, to mine these rules…
EP learns like BPTT but with local weight updates.
A new principle for optimizer selection improves training speed and performance.
The study analyzes deep linear networks from random initialization, capturing dynamics and hyperparameter effects.
Learning to remember long sequences remains a challenging task for recurrent neural networks. Register memory and attention mechanisms were both proposed to resolve the issue with either high computational cost to retain memory differentiability, or by discounting the RNN representation learning towards encoding shorte…
A novel Bayesian computation method using importance weighting improves numerical stability and performance.