A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
The paper provides convergence guarantees for multicalibration gradient boosting.
Paper proposes a new method for training nonconvex models.
Momentum Stochastic Gradient Descent (MSGD) algorithm has been widely applied to many nonconvex optimization problems in machine learning, e.g., training deep neural networks, variational Bayesian inference, and etc. Despite its empirical success, there is still a lack of theoretical understanding of convergence proper…
Listwise learning-to-rank methods form a powerful class of ranking algorithms that are widely adopted in applications such as information retrieval. These algorithms learn to rank a set of items by optimizing a loss that is a function of the entire set -- as a surrogate to a typically non-differentiable ranking metric.…
Random forests remain among the most popular off-the-shelf supervised machine learning tools with a well-established track record of predictive accuracy in both regression and classification settings. Despite their empirical success as well as a bevy of recent work investigating their statistical properties, a full and…
Neural networks extrapolate poorly in simple tasks but succeed in complex ones.
This work improves SGMs' convergence guarantees for semiconvex distributions with discontinuous gradients.
Student-teacher learning improves generalization with noisy inputs.
Television is an ever-evolving multi billion dollar industry. The success of a television show in an increasingly technological society is a vast multi-variable formula. The art of success is not just something that happens, but is studied, replicated, and applied. Hollywood can be unpredictable regarding success, as m…
New method uses robust estimators for Newton's method in empirical risk minimization.
Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner. However previous attempts toward developing online Monte-Carlo methods for LDA have little success, often by having much worse perplexity than their batch counterparts. We present a streaming Gibbs sampling (SGS) method, an onli…
Recently, over-parameterized neural networks have been extensively analyzed in the literature. However, the previous studies cannot satisfactorily explain why fully trained neural networks are successful in practice. In this paper, we present a new theoretical framework for analyzing over-parameterized neural networks …
A new method sorts models to find the best one with minimal risk.
Predicting startup success using Crunchbase data and deep learning.
AdaBoost's success explained through noise influence measure.
Graph neural networks improve solving linear optimization problems.
Belief Propagation algorithms are instruments used broadly to solve graphical model optimization and statistical inference problems. In the general case of a loopy Graphical Model, Belief Propagation is a heuristic which is quite successful in practice, even though its empirical success, typically, lacks theoretical gu…
The success of modern Artificial Intelligence (AI) technologies depends critically on the ability to learn non-linear functional dependencies from large, high dimensional data sets. Despite recent high-profile successes, empirical evidence indicates that the high predictive performance is often paired with low robustne…
Random Reshuffling outperforms Stochastic Gradient Descent in smooth convex optimization.
ReLU neural networks define piecewise linear functions of their inputs. However, initializing and training a neural network is very different from fitting a linear spline. In this paper, we expand empirically upon previous theoretical work to demonstrate features of trained neural networks. Standard network initializat…
Generative adversarial networks (GANs) have enjoyed much success in learning high-dimensional distributions. Learning objectives approximately minimize an -divergence (-GANs) or an integral probability metric (Wasserstein GANs) between the model and the data distribution using a discriminator. Wasserstein GANs en…
Improved sample complexity for diffusion models without needing empirical risk minimizers.
Distributional reinforcement learning (distributional RL) has seen empirical success in complex Markov Decision Processes (MDPs) in the setting of nonlinear function approximation. However, there are many different ways in which one can leverage the distributional approach to reinforcement learning. In this paper, we p…
The study provides a generalization bound for a family of implicit networks.
Despite the huge empirical success of deep learning, theoretical understanding of neural networks learning process is still lacking. This is the reason, why some of its features seem "mysterious". We emphasize two mysteries of deep learning: generalization mystery, and optimization mystery. In this essay we review and …
We analyze the probability density function (PDF) of waiting times between financial loss exceedances. The empirical PDFs are fitted with the self-excited Hawkes conditional Poisson process with a long power law memory kernel. The Hawkes process is the simplest extension of the Poisson process that takes into account h…
Multiplicative random cascade model naturally reproduces the intermittency or multifractality, which is frequently shown among hierarchical complex systems such as turbulence and financial markets. As described herein, we investigate the validity of a multiplicative hierarchical random cascade model through an empirica…
Recently, a unified model for image-to-image translation tasks within adversarial learning framework has aroused widespread research interests in computer vision practitioners. Their reported empirical success however lacks solid theoretical interpretations for its inherent mechanism. In this paper, we reformulate thei…
Indirect competition emerged from the complex organization of human societies, and knowledge of the existing network topology may aid in developing effective strategies for success. Here, we propose an agent-based model of competition with systems co-existing in a `small-world' social network. We show that within the r…
RLHF performs well despite violating social choice theory axioms.
In high dimensional settings, density estimation algorithms rely crucially on their inductive bias. Despite recent empirical success, the inductive bias of deep generative models is not well understood. In this paper we propose a framework to systematically investigate bias and generalization in deep generative models …
When machine predictors can achieve higher performance than the human decision-makers they support, improving the performance of human decision-makers is often conflated with improving machine accuracy. Here we propose a framework to directly support human decision-making, in which the role of machines is to reframe pr…
SSL framework identifies non-linear systems without labeled data.
New convergence rates for shuffling gradient methods without strong convexity.
Wasserstein GANs fail to approximate Wasserstein distance, leading to their success.
This work compares human feedback methods for reward learning in bandits.
New bound limits generalization gap for large models, independent of model complexity.
The use of momentum in stochastic gradient methods has become a widespread practice in machine learning. Different variants of momentum, including heavy-ball momentum, Nesterov's accelerated gradient (NAG), and quasi-hyperbolic momentum (QHM), have demonstrated success on various tasks. Despite these empirical successe…
Proposes a method to select features for deep learning in noisy, high-dimensional data.
The paper explores theoretical insights into WGANs for better understanding and stability.
The convergence rate and final performance of common deep learning models have significantly benefited from heuristics such as learning rate schedules, knowledge distillation, skip connections, and normalization layers. In the absence of theoretical underpinnings, controlled experiments aimed at explaining these strate…
Survey of data augmentation techniques for time series classification with neural networks.
A novel version of the Continuous-Time Random Walk (CTRW) model with memory is developed. This memory means the dependence between arbitrary number of successive jumps of the process, while waiting times between jumps are considered as i.i.d. random variables. The dependence was found by analysis of empirical histogram…
Paper analyzes convergence of DDPM for general distributions.
Improved adaptive algorithms for identifying the best arm in MABs with fixed budget.
New algorithm finds best feasible arm in grouped bandits.
This paper theoretically explains and validates a deep neural network approach to IV estimation.
Deep neural networks outperform traditional methods in high-dimensional classification.