ETM models improve efficiency in semi-supervised logistic regression.
problem Improving efficiency in logistic regression with limited labeled data.
method Developed exponential tilt mixture (ETM) models for semi-supervised estimation.
result ETM-based estimation demonstrates improved efficiency over supervised logistic regression.
Consider semi-supervised learning for classification, where both labeled and unlabeled data are available for training. The goal is to exploit both datasets to achieve higher prediction accuracy than just using labeled data alone. We develop a semi-supervised logistic learning method based on exponential tilt mixture m…
This paper considers the problem of measuring the credit risk in portfolios of loans, bonds, and other instruments subject to possible default under multi-factor models. Due to the amount of the portfolio, the heterogeneous effect of obligors, and the phenomena that default events are rare and mutually dependent, it is…
Iterative tilting fine-tunes diffusion models for reward-tilted distributions.
problem Fine-tuning diffusion models for reward-tilted distributions.
method Decomposes large reward tilts into smaller, tractable tilts via first-order Taylor expansion, avoiding backpropagation.
result Validated on a two-dimensional Gaussian mixture, achieving exact closed-form solutions.
We study tilting subweibull distributions and their tail behavior.
problem Understanding tail behavior of subweibull distributions.
method Alternative characterizations and conditions for tail behavior preservation.
result Conditions for tail behavior preservation after exponential tilting.
Extends ERM with exponential tilting for improved machine learning performance.
problem Improving machine learning models by adjusting loss weights for fairness and robustness.
method Tilted Empirical Risk Minimization (TERM) using exponential tilting.
result TERM can outperform traditional ERM and deliver competitive performance with state-of-the-art methods.
We construct the term structure of the (forward-looking, US market) equity risk premium from SPX option chains. The method is "model-light". Risk-neutral probability densities are estimated by fitting N-component Gaussian mixture models to option quotes, where N is a small integer (here 4 or 5). These densities are…
Study addresses RTB model performance drops due to distribution shifts.
problem Distribution shifts between training and target environments in RTB markets.
method Applies Exponential Tilt Reweighting Alignment (ExTRA) algorithm to estimate and correct model weights.
result Demonstrates improved RTB model performance using ExTRA algorithm.
New optimization method improves generalization across various tasks.
problem Improving zeroth-order optimization for better generalization.
method Exponential tilting objective to connect zeroth-order optimization with sharpness-aware minimization.
result Achieves better generalization compared to vanilla zeroth-order baselines.
Reweighting training data to better represent new tasks.
problem Deploying machine learning models to new tasks is challenging due to training data distribution.
method Formulate an exponential tilt distribution shift model and learn train data importance weights to minimize KL divergence.
result The learned train data weights improve target performance evaluation, fine-tuning, and model selection.
Parameters defined via General Estimating Equations (GEE) can be estimated by maximizing the Empirical Likelihood (EL). Newey and Smith (2004) have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n^-1) bias is small and that bias-corrected EL is higher-ord…
Parameters defined via general estimating equations (GEE) can be estimated by maximizing the empirical likelihood (EL). Newey and Smith [Econometrica 72 (2004) 219--255] have recently shown that this EL estimator exhibits desirable higher-order asymptotic properties, namely, that its O(n−1) bias is small and that …
The paper examines the tilted empirical risk's generalization and robustness under negative tilt.
problem The generalization error of machine learning algorithms under negative tilt.
method Uniform and information-theoretic bounds on the tilted generalization error under negative tilt.
result The tilted empirical risk's generalization error has a convergence rate of \(O(n^{-ε/(1+ε)})\).
Unified framework for training diffusion and flow models to sample from target distributions.
problem Training diffusion and flow models to sample from target distributions defined by exponential tilting.
method Unified framework combining stochastic optimal control and non-equilibrium thermodynamics perspectives.
result Unified bias-variance decompositions and theoretical support for adjoint-based methods.
Sharp large deviations and Gibbs conditioning for portfolio credit risk models.
problem Analyzing the risk of default in financial portfolios with dependent factors.
method Sharp large deviation estimates and conditional Bahadur-Rao estimates for threshold models with diverging latent factors.
result Conditioned on a large exceedance event, default indicators become asymptotically i.i.d., and loss-given-default is exponentially tilted.
Method adapts frozen models for few-shot tasks without training.
problem Deployment constraints limit model updates, necessitating new adaptation methods.
method Exponential tilting of latent distribution for inference.
result Method outperforms parameter-update methods across benchmarks.
Reweighted ALPS improves sampling from multimodal distributions using warm start points.
problem Sampling from multimodal distributions is hard due to exponential mixing times.
method Introduces Reweighted ALPS, a modified Annealed Leap-Point Sampler that uses warm start points.
result First polynomial-time bound for Re-ALPS in a general setting, under a natural assumption.
We propose the Bayesian bridge estimator for regularized regression and classification. Two key mixture representations for the Bayesian bridge model are developed: (1) a scale mixture of normals with respect to an alpha-stable random variable; and (2) a mixture of Bartlett--Fejer kernels (or triangle densities) with r…
Online TERM improves robustness and fairness in streaming data.
problem Streaming data's lack of worst-case fairness and robustness in ERM.
method Proposes an online TERM formulation to balance average-case accuracy with worst-case fairness and robustness.
result Negative tilting effectively suppresses outlier influence, positive tilting improves recall with minimal precision loss.
New method for Bayesian inference of Lévy-driven SDEs with jumps.
problem Bayesian inference for Lévy-driven SDEs is challenging due to discontinuities and heavy tails.
method Neural exponential tilting framework for variational inference.
result Accurately captures jump dynamics and reliable posterior inference in heavy-tailed regimes.
New method improves posterior sampling for complex data models.
problem Sampling from posterior distributions in high-dimensional data.
method Tilted transport technique combining denoising oracle and log-likelihood.
result Boosted posterior is strongly log-concave, facilitating easier sampling.
Paper establishes universal lower bounds and optimal rates for clustering sub-exponential mixture models.
problem Achieving optimal error rates in clustering sub-exponential mixture models.
method Establishes universal lower bounds and demonstrates iterative algorithms' optimality in sub-exponential mixture models.
result Iterative algorithms achieve the universal lower bound in sub-exponential mixture models.
Proposes a new prior for VAEs to improve out-of-distribution detection.
problem Probabilistic generative models struggle with out-of-distribution detection.
method Introduces an exponentially tilted Gaussian prior for VAEs.
result Achieves state-of-the-art results on ROC-AUC metric.
Paper proposes a method to improve MCMC sampling for energy-based models.
problem MCMC sampling of energy-based models is often not mixing in high-dimensional data.
method Proposes using a flow-based model as a backbone to correct the energy-based model, enabling mixing in latent space.
result MCMC sampling of the corrected EBM in the latent space mixes well and traverses modes in the data space.
Bayes-assisted confidence sequences improve efficiency for bounded means.
problem Efficient uncertainty quantification for bounded IID means without parametric assumptions.
method Bayesian working predictive model selects adaptive martingale updates maximizing predictive log-growth.
result Asymptotically log-optimal performance with informative priors reducing width and sampling effort.
The study analyzes how neural reward models learn features for policy optimization in a Gaussian single-index model.
problem Reward modeling in policy optimization and its impact on downstream value.
method Two-stage neural reward model: first learns hidden direction, then fits readout layer.
result For any feature-learning temperature above a dimension-free threshold, a constant fraction of neurons recover the hidden direction.
Research shows guidance in diffusion models does not sample from intended distribution, affecting boundary sampling.
problem Clarifying the misconception that guidance modifies the data distribution in diffusion models.
method Rigorous proof and fine-grained analysis of guidance dynamics in two cases: mixtures of compactly supported distributions and mixtures of Gaussians.
result Guidance leads to sampling more heavily from the boundary of the support of the conditional distribution as the parameter increases.
Develops conformal Bayes for two-sided censored Gaussian regression under label shift.
problem Prediction under label shift with censored responses.
method Combines posterior predictive tilting with weighted conformal calibration.
result Restores marginal coverage with smaller prediction sets.
New model tackles PU data with better accuracy.
problem Addressing positive and unlabeled data challenges.
method Double Exponential Tilting Model (DETM)
result DETM effectively handles selected at random PU data.
Researchers develop a method to generate diffusion-based samples from a tilted distribution.
problem Generating samples from a distribution that has been tilted by a parameter.
method Developed a plug-in estimator and proved Wasserstein bounds and TV-accuracy under certain conditions.
result The method is minimax-optimal and can be applied in various domains like finance and climate modeling.
Adaptive importance sampling techniques are widely known for the Gaussian setting of Brownian driven diffusions. In this work, we want to extend them to jump processes. Our approach relies on a change of the jump intensity combined with the standard exponential tilting for the Brownian motion. The free parameters of ou…
The study reveals the efficiency of sampling from tilted distributions.
problem Sampling from a tilted distribution of an unknown underlying distribution.
method Self-normalized importance sampling to characterize accuracy.
result Polynomial vs super-polynomial sample complexity for bounded vs unbounded distributions.
The paper introduces a new class of multivariate mixtures for actuarial applications.
problem Developing a new class of multivariate mixtures for actuarial calculations.
method Proposed a class of multivariate matrix-exponential affine mixtures with matrix-exponential marginals.
result Explicit calculations of actuarial quantities are possible due to the proposed class's properties.
A scalable algorithm for sampling and fine-tuning models using Tilt Matching.
problem Efficient sampling and fine-tuning of generative models.
method Tilt Matching, arising from a dynamical equation, minimizes variance and inherits regularity from stochastic interpolants.
result Empirically verified to be efficient and highly scalable, providing state-of-the-art results.
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
We study the problem of finding the smallest m such that every element of an exponential family can be written as a mixture of m elements of another exponential family. We propose an approach based on coverings and packings of the face lattice of the corresponding convex support polytopes and results from coding th…
EM algorithm converges exponentially fast for overspecified Gaussian mixtures.
problem Convergence of EM algorithm in overspecified Gaussian mixtures.
method Structured configuration of means and weights, strong convexity, Polyak-Łojasiewicz inequality, finite-sample analysis.
result Exponential convergence rate of EM algorithm in KL distance.
We construct an infinite-dimensional information manifold based on exponential Orlicz spaces without using the notion of exponential convergence. We then show that convex mixtures of probability densities lie on the same connected component of this manifold, and characterize the class of densities for which this mixtur…
We propose a novel exponentially-modified Gaussian (EMG) mixture residual model. The EMG mixture is well suited to model residuals that are contaminated by a distribution with positive support. This is in contrast to commonly used robust residual models, like the Huber loss or ℓ1, which assume a symmetric contami…
Optimizes antenna tilt for better QoS in cellular networks.
problem Hard to learn optimal antenna tilt policies in real networks due to risk and simulation gap.
method Uses off-policy Contextual Multi-Armed-Bandit (CMAB) techniques to learn from existing data.
result Trained policies show consistent improvements over existing logging policies.
Higher granularity in MoE models boosts expressivity exponentially.
problem Expressivity of Mixture-of-Experts models with varying granularity.
method Comparing models with different numbers of active experts (granularity).
result Exponential separation in network expressivity based on granularity.
Mutation graph of support τ-tilting modules over skew-gentle algebras is connected.
problem Understanding the structure of support τ-tilting modules over skew-gentle algebras.
method Introducing mutation of maximal rigid objects and using exchange triangles to define mutations of support τ-tilting modules.
result The mutation graph of support τ-tilting modules over a skew-gentle algebra is connected.
Optimizes portfolios with GM returns using convex optimization.
problem Maximizing expected exponential utility with GM asset returns.
method Formulated as a convex optimization problem.
result Optimal solutions found without sampling or scenarios.
Study shows MDA's effectiveness even when more components are assumed than in actual data.
problem Classification error in overspecified Mixture Discriminant Analysis.
method Two-component Gaussian mixture model, EM algorithm, theoretical analysis of convergence and error rates.
result EM algorithm converges exponentially fast to Bayes risk with suitable initialization.
We describe k-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expec…
We develop a general method for estimating a finite mixture of non-normalized models. Here, a non-normalized model is defined to be a parametric distribution with an intractable normalization constant. Existing methods for estimating non-normalized models without computing the normalization constant are not applicable …
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
problem Fitting k parametric functions to given data points without a generative model. method Gradient EM algorithm for agnostic mixtures of arbitrary parametric functions.
result Gradient EM converges exponentially to population loss minimizers with high probability.
We study the problem of learning an unknown mixture of k rankings over n elements, given access to noisy samples drawn from the unknown mixture. We consider a range of different noise models, including natural variants of the "heat kernel" noise framework and the Mallows model. For each of these noise models we giv…