We discuss the finite sample theoretical properties of online predictions in non-stationary time series under model misspecification. To analyze the theoretical predictive properties of statistical methods under this setting, we first define the Kullback-Leibler risk, in order to place the problem within a decision the…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “theoretical results”
The paper explores game-theoretic alignment of LLMs with human preferences, finding limitations and conditions.
problem Aligning LLMs with human preferences using game theory.
method Systematic study of payoff choices in a two-player zero-sum game for desirable alignment properties.
result Impossibility of preference matching in game-theoretic LLM alignment under standard assumptions.
LMs inevitably generate hallucinations, but can be made statistically negligible.
problem Inevitable hallucinations in language models.
method Probabilistic perspective and information theory.
result Hallucinations can be statistically negligible with sufficient training data and algorithm improvements.
Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacs.LG
New framework for learning from imbalanced data with theoretical guarantees.
problem Class imbalance in machine learning, especially in multi-class problems.
method Theoretical framework and new margin loss function for imbalanced classification.
result Proves strong -consistency of the proposed margin loss function.
We integrate information-theoretic concepts into the design and analysis of optimistic algorithms and Thompson sampling. By making a connection between information-theoretic quantities and confidence bounds, we obtain results that relate the per-period performance of the agent with its information gain about the enviro…
Theoretical analysis of deep neural networks for time series data.
problem Theoretical development for deep neural networks on temporally dependent observations is lacking.
method Established non-asymptotic bounds for prediction error of deep neural networks under mixing-type assumptions.
result Deep neural networks can model non-linear time series data with additional logarithmic factors due to dependence.
Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients evaluated on mini-batches are used as a replacement. In order to reduce the high vari…
Action chunking and data exploration improve behavior cloning in robotics.
problem Exponential errors in learning from demonstrations for continuous control tasks.
method Action chunking and exploratory data collection.
result Control-theoretic stability is key to improving imitation learning.
Survey on multiplayer bandits, highlighting theoretical gaps and future directions.
problem Theoretical advancements in multiplayer bandits lack practical implementation in real-world scenarios.
method Organizes and contextualizes existing literature on multiplayer bandits.
result Clear directions for future research in adapting theoretical algorithms to real-world situations.
Survey of deep learning methods for inverse problems, highlighting theoretical challenges.
problem Addressing theoretical gaps in deep learning for inverse problems.
method Survey of existing theoretical developments and open problems.
result Highlighting ongoing challenges in deep learning for inverse problems.
Unified framework improves meta-learning generalization bounds.
problem Limited sharpness of existing meta-generalization bounds.
method Unified information-theoretic derivation for single-step bounds.
result Unified bounds exhibit tighter scaling and computational advantages.
Dimension reduction and variable selection are performed routinely in case-control studies, but the literature on the theoretical aspects of the resulting estimates is scarce. We bring our contribution to this literature by studying estimators obtained via L1 penalized likelihood optimization. We show that the optimize…
We study the risk performance of distributed learning for the regularization empirical risk minimization with fast convergence rate, substantially improving the error analysis of the existing divide-and-conquer based distributed learning. An interesting theoretical finding is that the larger the diversity of each local…
In this paper we study the learnability of deep random networks from both theoretical and practical points of view. On the theoretical front, we show that the learnability of random deep networks with sign activation drops exponentially with its depth. On the practical front, we find that the learnability drops sharply…
We study the task of online boosting--combining online weak learners into an online strong learner. While batch boosting has a sound theoretical foundation, online boosting deserves more study from the theoretical perspective. In this paper, we carefully compare the differences between online and batch boosting, and pr…
By establishing a connection between bi-directional Helmholtz machines and information theory, we propose a generalized Helmholtz machine. Theoretical and experimental results show that given \textit{shallow} architectures, the generalized model outperforms the previous ones substantially.
The VAE's reconstruction ability is studied using PAC-Bayes theory.
problem Understanding the performance of VAEs for unseen data.
method PAC-Bayes theory is applied to analyze VAE's reconstruction error.
result Generalization bounds on VAE's reconstruction error are provided.
Theoretical study of random forests for nonlinear time series.
problem Theoretical justification for using random forests in time series modeling.
method Uniform concentration inequality for regression trees and random forests consistency proof.
result Consistency of random forests for nonlinear autoregressive processes.
Graph matching---aligning a pair of graphs to minimize their edge disagreements---has received wide-spread attention from both theoretical and applied communities over the past several decades, including combinatorics, computer vision, and connectomics. Its attention can be partially attributed to its computational dif…
The study initiates a theoretical analysis of dynamic benchmarking models.
problem Lack of theoretical foundation and empirical studies in dynamic benchmarks.
method Examined two realizations of dynamic benchmarking: sequential and hierarchical dependency models.
result Sequential dynamic benchmarks show initial performance improvement but can stall after three rounds due to label noise.
On the Generalization and Approximation Capacities of Neural Controlled Differential Equationsstat.ML
NCDEs improve predictions for irregular time series data.
problem Theoretical understanding of NCDEs' performance and irregular time series effects.
method Combining CDE theory and neural net complexity measures.
result Generalization bound and detailed sampling and approximation bias analysis.
M3PO improves model-based meta-RL with theoretical guarantees.
problem Improving sample efficiency in multi-task RL with theoretical guarantees.
method Extending Janner et al. (2019) theorems, proposing M3PO with performance guarantees.
result M3PO outperforms existing methods in continuous-control benchmarks.
Proposes a new information-theoretic framework for analyzing deep neural networks.
problem Difficulty in analyzing deep neural networks using existing theoretical frameworks.
method Introduces an information-theoretic framework with new notions of regret and sample complexity.
result Establishes sample complexity bounds for deep neural networks that are width-independent and linear in depth.
Develops a new robustness criterion for VAEs and provides theoretical guarantees.
problem Lack of formalization for robustness in VAEs.
method Introduces -robustness criterion and derives reconstruction margins.
result Derives theoretical guarantees for VAE robustness.
Theoretical analysis confirms non-conservative algorithms can converge to optimal policies.
problem Theoretical guarantees for non-conservative reinforcement learning algorithms.
method Theoretical analysis of Peng's Q() algorithm.
result Peng's Q() converges to an optimal policy under certain conditions.
New research shows existing information-theoretic methods can't establish minimax rates for gradient descent in stochastic convex optimization.
problem Establishing minimax rates for gradient descent in stochastic convex optimization using information-theoretic methods.
method Examined several information-theoretic frameworks including input-output mutual information bounds, conditional mutual information bounds, PAC-Bayes bounds, and their variants.
result Proved that none of the examined information-theoretic frameworks can establish minimax rates for gradient descent in stochastic convex optimization.
New model learns from random graph samples to estimate graph parameters.
problem Scalability issues in graph learning methods for large graphs.
method Develops a graph classification model working on randomly sampled subgraphs.
result Validates mini-batch learning on graphs and provides generalization bounds.
Recovery of low-rank matrices from a small number of linear measurements is now well-known to be possible under various model assumptions on the measurements. Such results demonstrate robustness and are backed with provable theoretical guarantees. However, extensions to tensor recovery have only recently began to be st…
We provide a general theoretical analysis of expected out-of-sample utility, also referred to as decision-theoretic classification, for non-decomposable binary classification metrics such as F-measure and Jaccard coefficient. Our key result is that the expected out-of-sample utility for many performance metrics is prov…
Paper analyzes self-supervised image denoising with denatured data.
problem Understanding the performance of self-supervised image denoising with denatured data.
method Theoretical analysis and numerical experiments on a denoising algorithm.
result Theoretical analysis shows the algorithm finds desired solutions to the optimization problem.
This work analyzes Fréchet regression using comparison geometry, providing theoretical and practical insights.
problem Analyzing data on complex structures like manifolds and graphs.
method Theoretical analysis through comparison geometry, focusing on existence, uniqueness, and stability of the Fréchet mean.
result Key results on the existence, uniqueness, and stability of the Fréchet mean, along with statistical guarantees for nonparametric regression.
The paper explores the information-theoretic nature of excess risk in machine learning.
problem Understanding the excess risk in machine learning models.
method Formulates the minimax excess risk as a zero-sum game and modifies it to allow swapping of the order of play.
result Proves that under certain conditions, the duality gap is zero, allowing for the application of Bayesian results to provide bounds on minimax excess risk.
Paper explains why small-loss criterion works for learning from noisy labels.
problem Learning from noisy labels in deep learning with limited labeled data.
method Theoretical analysis and reformulation of the small-loss criterion.
result Theoretical explanation and reformulation of the small-loss criterion.
In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., ordinary supervised learning). Hitherto, neither theoretical nor experimental analysis has been given to explain this phenomenon. In …
Understanding theoretical properties of deep and locally connected nonlinear network, such as deep convolutional neural network (DCNN), is still a hard problem despite its empirical success. In this paper, we propose a novel theoretical framework for such networks with ReLU nonlinearity. The framework explicitly formul…
This research provides theoretical guarantees for hyperparameter estimation in complex network dynamical systems.
problem Theoretical guarantees for hyperparameter estimation in large, inhomogeneous complex network dynamical systems.
method Formulating the system's evolution in a measure transport perspective, proposing a theoretical framework for estimating hyperparameters with mean-type observations.
result A nonasymptotic bound for the deviation of hyperparameter estimates in inhomogeneous complex network dynamical systems with respect to network population size.
We call a group FJ if it satisfies the - and -theoretic Farrell-Jones conjecture with coefficients in . We show that if is FJ, then the simple Borel conjecture (in dimensions ) holds for every group of the form . If in addition , which is true for …
Theoretical justification for asymmetric actor-critic algorithms in reinforcement learning.
problem Lack of precise theoretical justification for asymmetric actor-critic algorithms in reinforcement learning.
method Adapting a finite-time convergence analysis to the asymmetric actor-critic setting with linear function approximators.
result A finite-time bound reveals that the asymmetric critic eliminates aliasing errors in the agent state.
Extended analysis of Q-learning's efficiency, matching optimal regret.
problem Theoretical guarantees of Q-learning's efficiency and optimal regret.
method Survey of related research, detailed proof reasoning.
result Q-learning with UCB exploration achieves sample efficiency matching optimal regret.
The paper explores stability and generalization of deep GCNs.
problem Understanding the stability and generalization of deep GCNs from a theoretical perspective.
method Theoretical analysis of stability and generalization properties of deep GCNs.
result The stability and generalization of deep GCNs are influenced by the maximum absolute eigenvalue of the graph filter operators and the depth of the network.
Paper proves stacking ensembling is effective and proposes a new family of stacked generalizations.
problem Lack of theoretical guarantees for stacking ensembling methods.
method Proves novel theoretical result and proposes a new family of stacked generalizations.
result Proves stacking ensembling is effective and proposes a new family of stacked generalizations.
We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in the empirical risk with model compression. We first prove that model compression reduces an information-theoretic bound on the generalizati…
Unpublished results of S Straus and W Browder state that two notions of homotopy equivalence for manifolds with smooth group actions - isovariant and equivariant - often coincide under a condition called the Gap Hypothesis; the proofs use deep results in geometric topology. This paper analyzes the difference between th…
This paper investigates the theory of robustness against adversarial attacks. It focuses on the family of randomization techniques that consist in injecting noise in the network at inference time. These techniques have proven effective in many contexts, but lack theoretical arguments. We close this gap by presenting a …
Discrimination-aware classification is receiving an increasing attention in data science fields. The pre-process methods for constructing a discrimination-free classifier first remove discrimination from the training data, and then learn the classifier from the cleaned data. However, they lack a theoretical guarantee f…
Theoretical study shows AI models can recover from contaminated training data.
problem Data contamination in AI training can degrade model performance.
method Theoretical analysis and experiments on various data types.
result Models converge to true distribution under mild conditions, with rate dependent on real data fraction.
Bayesian MAML outperforms MAML in meta learning tasks with theoretical guarantees.
problem Theoretical understanding of Bayesian MAML's superiority over MAML.
method Comparison of meta test risks between Bayesian MAML and MAML in meta linear regression.
result Bayesian MAML has provably lower meta test risks than MAML in both distribution agnostic and linear centroid cases.
Study improves theoretical understanding of Bayesian deep learning for classification tasks.
problem Theoretical gap in understanding Bayesian approaches in deep learning for classification.
method PAC-Bayes bounds techniques and Spike-and-Slab priors for sparse deep learning.
result Established non-asymptotic results for prediction error, achieving minimax optimal rates.