Understanding theoretical properties of deep and locally connected nonlinear network, such as deep convolutional neural network (DCNN), is still a hard problem despite its empirical success. In this paper, we propose a novel theoretical framework for such networks with ReLU nonlinearity. The framework explicitly formul…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “theoretical frameworks”
The paper analyzes frameworks for integrating sustainability into investment decisions.
problem Understanding how ESG factors influence investment choices.
method Examined and analyzed various theoretical frameworks including Behavioral Finance, Modern Portfolio, and Risk Management.
result Investors increasingly integrate ESG factors to optimize financial outcomes and societal goals.
Proposes a new information-theoretic framework for analyzing deep neural networks.
problem Difficulty in analyzing deep neural networks using existing theoretical frameworks.
method Introduces an information-theoretic framework with new notions of regret and sample complexity.
result Establishes sample complexity bounds for deep neural networks that are width-independent and linear in depth.
A framework combines unsupervised and semi-supervised AD using synthetic anomalies.
problem Improving anomaly detection in both unsupervised and semi-supervised settings.
method Proposes a new framework that uses both known and synthetic anomalies for training.
result Synthetic anomalies improve anomaly modeling in low-density regions and provide optimal convergence guarantees.
Proposes a new theoretical framework for PbRL that requires less human feedback.
problem Lack of theoretical work capturing practical PbRL frameworks.
method Introduces a reward-agnostic PbRL framework that acquires exploratory trajectories before human feedback.
result Demonstrates improved sample complexity for learning optimal policies in linear and low-rank MDPs.
Paper develops a framework to optimize neural networks using weighted metrics.
problem Discrepancy between maximizing weighted classification scores and minimizing loss function.
method Formalizes weighted classification metrics and constructs corresponding losses.
result Framework includes well-established approaches like cost-sensitive learning and weighted cross entropy.
A game-theoretic framework identifies influential hyperparameters for neural networks.
problem Understanding which hyperparameters are most important for neural network performance.
method Employing Shapley Effects for global sensitivity analysis and Pareto front sets for identifying effective configurations.
result Reveals which hyperparameters are most influential for different objectives in neural networks.
Unified framework for removing unwanted information from machine learning models.
problem Removing undesirable features or data points from machine learning models while preserving utility.
method Information-theoretic regularization approach for data point and feature unlearning.
result Unified mathematical framework with provable guarantees for both data point and feature unlearning.
Balancing the Scales: A Theoretical and Algorithmic Framework for Learning from Imbalanced Datacs.LG
New framework for learning from imbalanced data with theoretical guarantees.
problem Class imbalance in machine learning, especially in multi-class problems.
method Theoretical framework and new margin loss function for imbalanced classification.
result Proves strong -consistency of the proposed margin loss function.
New method uses dynamic programming for meta continual learning.
problem Challenges of generalization and catastrophic forgetting in sequential learning.
method Developed a theoretical framework using dynamic programming for meta continual learning.
result Theoretical and practical method achieves better accuracy than existing methods.
Unified framework improves diffusion model rewards without full trajectories.
problem Limited theoretical understanding of guided diffusion samplers.
method Developed a unified algorithmic and theoretical framework for diffusion guidance and reward-guided diffusion.
result Framework shows CFG decreases expected reciprocal of classifier probability.
Theoretical framework for data augmentation in finance improves portfolio construction.
problem Improving portfolio construction in speculative markets.
method Developed a theoretical framework for data augmentation and regularization in deep learning for finance.
result A simple noise injection algorithm improves portfolio construction over no noise.
Develops a framework for consistent loss functions with variable transformations.
problem Lack of theoretical understanding of variable transformations in consistent loss functions.
method Formal characterizations of consistency for transformed loss functions in two cases: realization and prediction variables.
result Establishes new identifiable and elicitable functionals for complex predictive tasks.
Framework for understanding overfitting and underfitting using information theory.
problem Understanding and preventing overfitting and underfitting in machine learning.
method Information-theoretic framework measuring algorithm capacity and dataset information transfer.
result Upper-bounding algorithm capacity and establishing its relationship to machine learning quantities.
The paper provides a theoretical framework for machine learning.
problem Lack of rigorous theory guiding machine learning experiments.
method Bayesian statistics and Shannon's information theory.
result Theoretical insights applicable across various machine learning settings.
We introduce a framework for analyzing transductive combination of Gaussian process (GP) experts, where independently trained GP experts are combined in a way that depends on test point location, in order to scale GPs to big data. The framework provides some theoretical justification for the generalized product of GP e…
A framework for ranking with abstention, offering theoretical guarantees and practical effectiveness.
problem Making predictions with limited cost when uncertain.
method Introduces a novel ranking framework with abstention, analyzing theoretical consistency bounds.
result Extensive theoretical analysis including -consistency bounds for linear and neural network models.
Model-based reinforcement learning (RL) is considered to be a promising approach to reduce the sample complexity that hinders model-free RL. However, the theoretical understanding of such methods has been rather limited. This paper introduces a novel algorithmic framework for designing and analyzing model-based RL algo…
A Theoretical Framework for LLM Fine-tuning Using Early Stopping for Non-random Initializationstat.ML
Theoretical framework explains why few epochs are enough for LLM fine-tuning.
problem Understanding why few epochs are sufficient for LLM fine-tuning.
method Combining early stopping theory with attention-based Neural Tangent Kernel (NTK) for LLMs.
result Formalizes convergence rate of attention-based fine-tuning with respect to sample size.
Paper analyzes history-based RL methods for MDPs, introduces a theoretical framework and practical algorithm.
problem Improving RL performance in MDPs using history-based features.
method Theoretical framework for history-based RL, practical algorithm design.
result Practical RL algorithm shows effectiveness on continuous control tasks.
Deep neural networks are often used to implement powerful generative models for real-world data. Notable applications include image denoising, as well as other classical inverse problems like compressed sensing and super-resolution. To provide a rigorous but simplified analysis of generative models, in this work, we in…
Testing Ambiguity and Machina Preferences Within a Quantum-theoretic Framework for Decision-makingecon.GN
The Machina thought experiments pose to major non-expected utility models challenges that are similar to those posed by the Ellsberg thought experiments to subjective expected utility theory (SEUT). We test human choices in the `Ellsberg three-color example', confirming typical ambiguity aversion patterns, and the `Mac…
Unified framework for OOD detection and generalization using graph theory.
problem Challenges in out-of-distribution (OOD) generalization and detection in real-world machine learning models.
method Graph-theoretic framework to jointly tackle OOD generalization and detection.
result Empirical validation of theoretical underpinnings with competitive performance.
The paper improves reinforcement learning stability and efficiency with a new theoretical framework.
problem Stability and efficiency in reinforcement learning, especially in data-scarce scenarios.
method Theoretical framework using resampled - and -statistics to model experience replay, applied to policy evaluation and kernel ridge regression.
result Significant improvements in stability and efficiency, particularly in data-scarce scenarios.
Unified framework for active learning problems using information theory.
problem Combining level set estimation and Bayesian optimization.
method Information-theoretic criterion and acquisition function.
result Unified framework achieves state-of-the-art performance.
We discuss the finite sample theoretical properties of online predictions in non-stationary time series under model misspecification. To analyze the theoretical predictive properties of statistical methods under this setting, we first define the Kullback-Leibler risk, in order to place the problem within a decision the…
Unified framework connects EI and information-theoretic acquisition functions.
problem Distinguish between Expected Improvement and information-theoretic acquisition functions.
method Introduces Variational Entropy Search (VES) to unify EI and information-theoretic approaches.
result EI can be seen as a variational inference approximation of Max-value Entropy Search (MES).
Paper analyzes how contrastive learning structures learned representations.
problem Understanding the structure of learned representations in contrastive learning.
method Kernel-based contrastive learning framework (KCL) and statistical dependency viewpoint.
result Theoretical upper bound and generalization error bound for KCL.
Labour productivity distribution (dispersion) is studied both theoretically and empirically. Superstatistics is presented as a natural theoretical framework for productivity. The demand index is proposed within this framework as a new business index. Japanese productivity data covering small-to-medium to large firm…
The paper explores game-theoretic alignment of LLMs with human preferences, finding limitations and conditions.
problem Aligning LLMs with human preferences using game theory.
method Systematic study of payoff choices in a two-player zero-sum game for desirable alignment properties.
result Impossibility of preference matching in game-theoretic LLM alignment under standard assumptions.
Unified framework improves meta-learning generalization bounds.
problem Limited sharpness of existing meta-generalization bounds.
method Unified information-theoretic derivation for single-step bounds.
result Unified bounds exhibit tighter scaling and computational advantages.
Many binary classification problems minimize misclassification above (or below) a threshold. We show that instances of ranking problems, accuracy at the top or hypothesis testing may be written in this form. We propose a general framework to handle these classes of problems and show which known methods (both known and …
Theory of learning of high-dimensional controlled non-linear dynamical systems (I): models and methodscond-mat.dis-nn
Neural ODEs provide a framework for studying the training dynamics of neural networks.
problem Training dynamics of neural networks
method Dynamical mean field theory
result Derive learning curves in the high-dimensional limit
New model learns from random graph samples to estimate graph parameters.
problem Scalability issues in graph learning methods for large graphs.
method Develops a graph classification model working on randomly sampled subgraphs.
result Validates mini-batch learning on graphs and provides generalization bounds.
Paper addresses theoretical risks in neural MCCFR, proposing Robust Deep MCCFR for improved performance.
problem Theoretical risks in neural MCCFR, especially in large games.
method Adaptive framework with selective component deployment, including target networks, exploration, and variance-aware training.
result Robust Deep MCCFR achieves significant exploitability improvements in both Kuhn and Leduc Poker.
Imitation learning trains a policy from expert demonstrations. Imitation learning approaches have been designed from various principles, such as behavioral cloning via supervised learning, apprenticeship learning via inverse reinforcement learning, and GAIL via generative adversarial learning. In this paper, we propose…
Approximate inference algorithm is one of the fundamental research fields in machine learning. The two dominant theoretical inference frameworks in machine learning are variational inference (VI) and Markov chain Monte Carlo (MCMC). However, because of the fundamental limitation in the theory, it is very challenging to…
Mathematical framework for differential machine learning in finance.
problem Theoretical assumptions in financial models and their impact on machine learning algorithms.
method Rigorous mathematical framework for differential machine learning in finance.
result Theoretical grounding enhances the predictive capabilities of neural networks in financial applications.
New framework minimizes model complexity for improved few-shot learning.
problem Empirical benefits of pre-training scale with data size but lack theoretical explanation.
method Complexity Minimization framework for meta-representation learning.
result Theoretical analysis shows error rate improves with more meta-training data.
New research shows existing information-theoretic methods can't establish minimax rates for gradient descent in stochastic convex optimization.
problem Establishing minimax rates for gradient descent in stochastic convex optimization using information-theoretic methods.
method Examined several information-theoretic frameworks including input-output mutual information bounds, conditional mutual information bounds, PAC-Bayes bounds, and their variants.
result Proved that none of the examined information-theoretic frameworks can establish minimax rates for gradient descent in stochastic convex optimization.
Unified framework for FDR control in knockoffs, validating Gaussian knockoffs.
problem Asymptotic FDR control in knockoffs with user-specified distributions.
method Unified theoretical framework, three conditions on approximate knockoff statistics, Gaussian knockoffs generator based on moments matching.
result Gaussian knockoffs generator achieves asymptotic FDR control.
We present a simple theoretical framework, and corresponding practical procedures, for comparing probabilistic models on real data in a traditional machine learning setting. This framework is based on the theory of proper scoring rules, but requires only basic algebra and probability theory to understand and verify. Th…
Unified framework for stable RL learning with theoretical guarantees.
problem Lack of systematic theoretical principles guiding RL post-training methods.
method Unified theoretical framework for policy-gradient estimators and optimization algorithms.
result Establishes unbiasedness, variance expressions, and convergence guarantees.
New framework evaluates model explanations based on decision task improvement.
problem Evaluation of model explanations often misses practical value.
method Decision-theoretic framework quantifying three key values.
result Provides benchmarks and interprets human-AI decision support.
A new gradient flow framework for distributionally robust optimization.
problem Optimizing under uncertainty with worst-case distributional constraints.
method Gradient flow theory applied to distributionally robust optimization.
result Practical algorithms for sampling from worst-case distributions.
Improves continual learning with theoretical guarantees and a new algorithm.
problem Learning incremental tasks with dynamic data distributions.
method Contrastive and distillation losses with theoretical performance guarantees.
result Theoretical performance bounds and improved continual learning performance.
Continuous time framework for discrete data denoising models.
problem Efficient training and sampling for discrete data denoising models.
method Formulated as Continuous Time Markov Chains (CTMCs), efficient training using continuous time ELBO, high-dimensional CTMC simulation, novel theoretical error bound.
result Continuous time treatment enables novel theoretical error bound between generated and true data distributions.
Fairness-aware classification is receiving increasing attention in the machine learning fields. Recently research proposes to formulate the fairness-aware classification as constrained optimization problems. However, several limitations exist in previous works due to the lack of a theoretical framework for guiding the …