A tutorial on variational inference for high-dimensional models.
problem Approximating marginal likelihood and posterior in Bayesian models.
method Parametric approach to variational inference.
result Variational inference is now preferred for high-dimensional models and large datasets.
The paper examines Wiener process for LID estimation methods.
problem Estimating local intrinsic dimension in high-dimensional datasets.
method Investigates recent LID estimation methods from a Wiener process perspective.
result Explains how methods behave under non-ideal conditions.
Large deviations theory applied to policy gradient methods.
problem Understanding convergence of policy gradient methods in reinforcement learning.
method Large deviation rate function and contraction principle from large deviations theory.
result Convergence properties of policy gradient methods can be extended to various policy parametrizations.
The paper investigates model collapse in language models from a probabilistic perspective.
problem Understanding and preventing model collapse in language model training.
method Investigates recursive parametric model training from a probabilistic standpoint, characterizing conditions for model collapse and proposing mitigation strategies.
result Progressively increasing sample size is necessary to prevent model collapse, with a superlinear growth rate required in the asymptotic regime.
Paper introduces FNM framework for learning finite-dimensional parametrized models.
problem Efficiently learning finite-dimensional parametrized models from limited data.
method Fourier Neural Mappings (FNMs) framework for operator learning.
result End-to-end learning of PtO maps can be less data-efficient than learning the solution operator first.
A universal LSTM model outperforms asset-specific models in forecasting stock volatilities.
problem Forecasting stock volatilities across different assets.
method Trained an LSTM network on a pooled dataset of liquid stocks to forecast daily realized volatilities.
result The LSTM model consistently outperforms other asset-specific parametric models in volatility forecasting.
RL methods applied to option pricing using modified QLBS and RLOP models.
problem Applying reinforcement learning to price options accurately.
method Developed modified QLBS and RLOP models, implemented RL learning algorithm with neural networks.
result Optimal hedging strategies learned by RL outperform baseline models.
Representation costs in data science: Unifying function-space views of parametric methods
problem Analyzing representation costs of parametric data-fitting methods
method Developing a general framework for analyzing representation costs through parameter-space regularizers
result Proving that many natural results hold in this abstract setting, including representer theorems for parametric methods on their native spaces
We construct a map from the suspension G-spectrum ΣG∞M of a smooth compact G-manifold to the equivariant A-theory spectrum AG(M), and we show that its fiber is, on fixed points, a wedge of stable h-cobordism spectra. This map is constructed as a map of spectral Mackey functors, which is compatible …
We consider solution of stochastic storage problems through regression Monte Carlo (RMC) methods. Taking a statistical learning perspective, we develop the dynamic emulation algorithm (DEA) that unifies the different existing approaches in a single modular template. We then investigate the two central aspects of regres…
Non-parametric approaches for analyzing network data based on exchangeable graph models (ExGM) have recently gained interest. The key object that defines an ExGM is often referred to as a graphon. This non-parametric perspective on network modeling poses challenging questions on how to make inference on the graphon und…
A new MFG framework for evolving clusters from Gaussian mixtures.
problem Evolutionary clustering of time-dependent Gaussian mixtures.
method Control-theoretic framework based on Mean Field Games (MFG) with coupled HJB and Fokker-Planck systems.
result MFG dynamics recover classical EM algorithm trajectories with mass conservation.
Non-parametric bootstrap improves robust portfolio and trading strategy optimization.
problem Mitigating uncertainty in expected returns and covariances in financial decision-making.
method Non-parametric bootstrap framework for robust optimization without distributional assumptions.
result Improved out-of-sample performance with smoother, more stable results.
We consider grouping as a general characterization for problems such as clustering, community detection in networks, and multiple parametric model estimation. We are interested in merging solutions from different grouping algorithms, distilling all their good qualities into a consensus solution. In this paper, we propo…
Paper connects AJ conjecture and colored Jones polynomial potential function.
problem Relationship between A-polynomial and colored Jones polynomial. method Connects AJ conjecture and colored Jones polynomial potential function.
result Establishes connection between A-polynomial and colored Jones polynomial potential function. Unified approach for quantum and classical learning from evaluation oracles.
problem Learning from evaluation oracles in quantum and classical settings.
method Inspired by Kearns' SQ and Valiant's weak evaluation oracle, a unified framework is established.
result Characterizes query complexity for learning linear function classes and extends learnability results for quantum circuits.
Extends Demographic Parity for fairer wage predictions with expert knowledge.
problem Inadequate fairness metrics limit user domain knowledge and ignore intersectional fairness.
method Develops a parametric method to incorporate expert knowledge in fair predictions.
result Offers a robust solution for real-life applications with limited data and spending constraints.
Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for a reinforcement learning problem, which extends the standard Q-learning approach to incorporate a two-stream framework of reward processing with biases biologically associated with several neurolo…
High-dimensional data models, often with low sample size, abound in many interdisciplinary studies, genomics and large biological systems being most noteworthy. The conventional assumption of multinormality or linearity of regression may not be plausible for such models which are likely to be statistically complex due …
The author suggests using non-Euclidean geometry for psychometric models.
problem Current psychometric models lack geometric insights.
method Illustrates how non-Euclidean geometry can be applied to psychometrics.
result Geometric concepts may improve psychometric model understanding.
The MICZ-Kepler orbits are the non-colliding orbits of the MICZ Kepler problems (the magnetized versions of the Kepler problem). The oriented MICZ-Kepler orbits can be parametrized by the canonical angular momentum L and the Lenz vector A, with the parameter space consisting of the pairs of 3D vecto…
Gradient descent in neural networks analyzed using RKBS for broader applicability.
problem Analyzing neural network training in the over-parametrized limit.
method Constructing an exact power-series representation of neural networks in RKBS, proving replicability of gradient descent sequences.
result Gradient descent sequences can be exactly replicated by regularized sequential learning in RKBS, providing new theoretical insights.
We demonstrate that almost all non-parametric dimensionality reduction methods can be expressed by a simple procedure: regularized loss minimization plus singular value truncation. By distinguishing the role of the loss and regularizer in such a process, we recover a factored perspective that reveals some gaps in the c…
We study the relationship between online Gaussian process (GP) regression and kernel least mean squares (KLMS) algorithms. While the latter have no capacity of storing the entire posterior distribution during online learning, we discover that their operation corresponds to the assumption of a fixed posterior covariance…
Proposes MvTPMSVM to improve multiview learning with reduced computational complexity.
problem Challenges in multiview learning, especially with heteroscedastic noise.
method Introduces MvTPMSVM, a parametric margin SVM model that avoids matrix inversions.
result Demonstrates superior generalization compared to baseline models.
Paper proposes Nyström sketches for better adaptive compressive learning.
problem Improving adaptability of sketching for compressive learning.
method Data-dependent Nyström approximation for mean embedding.
result Excess risk can be controlled with geometric assumption.
Theoretical and empirical taxonomy of imbalance in binary classification.
problem Class imbalance degrades binary classification performance.
method Proposed a principled framework based on three scales: imbalance coefficient, sample-dimension ratio, and intrinsic separability. Derived closed-form Bayes errors and analyzed degradation across models.
result The triplet (η, κ, Δ) provides a model-agnostic explanation of imbalance-induced deterioration.
Study non-parametric frequency-domain system identification from finite samples.
problem Frequency-domain system identification from limited data.
method Empirical Transfer Function Estimate (ETFE) under sub-Gaussian colored noise and stability assumptions.
result ETFE estimates are concentrated around true values with a finite-sample rate of Ntot−1/3 for all frequencies in the H∞ norm. In this sequel we extend the derivation of the third order helicity to magnetic fields supported on unlinked domains in 3-space. The formula is expressed in terms of generators of the deRham cohomology of the configuration space of three points in R3, which is a more practical domain from the perspective of applica…
Deep learning has arguably achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex dependency between input features and labels. While neural networks have a long history, recent advances have greatly improved their performance in…
Quantum models avoiding barren plateaus can also be efficiently simulated classically.
problem Understanding the limitations of barren plateaus in quantum computing.
method Analyzing commonly used models and their ability to be simulated classically.
result Many quantum models with barren plateau-free landscapes can also be efficiently simulated classically.
The paper provides a statistical decision-theoretical derivation of the Two-Stage approach for parameter estimation.
problem Theoretical justification for the Two-Stage approach in situations where likelihood is difficult to evaluate.
method Statistical decision-theoretical derivation leading to Bayesian and Minimax estimators.
result The Two-Stage approach is justified theoretically and applied to independent and identically distributed samples.
The paper introduces a DRM for causal inference, offering a flexible method to analyze counterfactual distributions.
problem Estimating mean causal effects is limited; a distributional perspective is needed for a more thorough understanding.
method The paper employs a semiparametric density ratio model (DRM) with an empirical likelihood (EL) approach to estimate counterfactual distribution functions.
result The DRM framework enables direct and transparent causal inference from a distributional perspective, validated by numerical studies.
This paper is a continuation of the previous paper of the author[M]. We show that an affine deformation space of a hyperbolic surface of type (g,b) can be parametrized by Margulis invariants and affine twist parameters with a certain decomposition of the surface, which are associated with the Fenchel-Nielsen coordinate…
New approach improves multi-head attention by making heads less similar.
problem Multi-head attention can lead to similar features, reducing model expressiveness.
method Proposes a non-parametric approach using Bayesian techniques to make heads repel each other.
result Improves feature diversity, leading to better representations and performance.
Spray-invariant sets maintain geodesics on infinite-dimensional manifolds.
problem Geodesic preservation in infinite-dimensional manifolds.
method Definition of spray-invariant sets and analysis of their properties.
result Different geometric properties of spray-invariant sets based on their regularity.
The paper provides stability guarantees for non-parametric maximum likelihood estimation using statistical mechanics.
problem Non-parametric maximum likelihood estimation and Gaussian mixture models.
method Statistical mechanics analysis to establish stability guarantees for NPMLE.
result High probability upper bounds on the Kullback-Leibler divergence between NPMLE estimators and the true density.
This paper describes a pattern recognition approach aiming to estimate fuel cell duration time from electrochemical impedance spectroscopy measurements. It consists in first extracting features from both real and imaginary parts of the impedance spectrum. A parametric model is considered in the case of the real part, w…
The Expectation-Maximization (EM) algorithm is one of the most popular methods used to solve the problem of parametric distribution-based clustering in unsupervised learning. In this paper, we propose to analyze a generalized EM (GEM) algorithm in the context of Gaussian mixture models, where the maximization step in t…
Let W -> A^2 be the universal Weierstrass family of cubic curves over C. For each N >= 2, we construct surfaces parametrizing the three standard kinds of level N structures on the smooth fibers of W. We then complete these surfaces to finite covers of A^2. Since W -> A^2 is the versal deformation space of a cusp singul…
This work tackles manifold regression onto hyperbolic space for tree classification and taxonomy extension.
problem Performing manifold-valued regression onto an hyperbolic space for tree classification and taxonomy extension.
method Formulated as a manifold regression task in hyperbolic space, proposed a parametric deep learning model and a non-parametric kernel method.
result Hyperbolic-based estimators significantly outperform Euclidean space methods in taxonomy expansion.
Researchers approximate conditional expectation operators using kernel methods.
problem Statistical approximation of conditional expectation operators under minimal assumptions.
method Modifying the domain of the operator, approximating it by Hilbert-Schmidt operators in a reproducing kernel Hilbert space.
result The nonparametric estimate of the operator converges to a specific limiting object.
New estimates for the population risk are established for two-layer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the over-parametrized regime when the network size is much larger than the size of…
New criterion selects optimal number of clusters based on stability.
problem Challenges in selecting optimal number of clusters in non-parametric clustering.
method Proposes a stability-based validation criterion combining between-cluster and within-cluster stability.
result Empirically demonstrates effectiveness in selecting optimal number of clusters.
Study measures invariant DH on twisted moduli spaces.
problem Characterizing invariant measures on twisted moduli spaces.
method Quasi-Hamiltonian perspective, Duistermaat-Heckman measures, Fourier coefficients.
result Characterized invariant Duistermaat-Heckman measures on twisted moduli spaces.
We propose the Wasserstein-Fourier (WF) distance to measure the (dis)similarity between time series by quantifying the displacement of their energy across frequencies. The WF distance operates by calculating the Wasserstein distance between the (normalised) power spectral densities (NPSD) of time series. Yet this ratio…
We present a framework for incorporating prior information into nonparametric estimation of graphical models. To avoid distributional assumptions, we restrict the graph to be a forest and build on the work of forest density estimation (FDE). We reformulate the FDE approach from a Bayesian perspective, and introduce pri…
Score-based diffusion models achieve optimal error bounds under non-parametric assumptions.
problem Improving the minimax optimality of score-based diffusion models.
method Kernel-based score estimation and early stopping strategy.
result Achieves minimax optimal error bounds under sub-Gaussian and Sobolev space assumptions.