Bayesian model for discrete data with conditional transformations.
problem Handling discrete ordinal and count data with excess zeros.
method Bayesian framework with conditional transformation functions and modular MCMC algorithm.
result Flexible modeling of linear and nonlinear covariate effects for ordinal and count data.
GTMs model complex multivariate data with varying conditional independencies.
problem Modeling multivariate data with intricate marginals and complex dependency structures.
method Semiparametric approach using penalized splines and lasso regularization.
result GTMs accurately learn complex dependencies and identify conditional independencies.
Deep conditional transformation models unify interpretable and complex predictors.
problem Challenging to learn conditional CDFs in high-dimensional settings.
method Unified deep learning framework for interpretable and complex predictors.
result Efficacy demonstrated through numerical experiments and applications.
Regression models for supervised learning problems with a continuous target are commonly understood as models for the conditional mean of the target given predictors. This notion is simple and therefore appealing for interpretation and visualisation. Information about the whole underlying conditional distribution is, h…
Enhances Transformers for better risk assessment in finance.
problem Transformer models lack sensitivity to extreme financial losses.
method Integrates Loss-at-Risk function with Value at Risk (VaR) and Conditional Value at Risk (CVaR).
result Improves risk prediction and management in financial datasets.
The paper models musical motif transformations in Beethoven's works.
problem Understanding how motifs transform in symbolic music.
method Developed a probabilistic framework using Conditional Random Fields.
result Identified patterns of motif transformations and their co-occurrences.
ACE models allow flexible conditioning and prediction of latent variables.
problem Lack of flexibility in conditioning and prediction of latent variables in probabilistic models.
method Introduces Amortized Conditioning Engine (ACE) that explicitly represents latent variables and allows runtime conditioning and prediction.
result ACE models outperform existing methods in diverse tasks like image completion, classification, Bayesian optimization, and simulation-based inference.
Transformers can learn Markov processes with constant depth, surprising results.
problem Understanding how transformers learn context in Markov processes.
method Empirical study and theoretical analysis of attention-based transformers on Markov data.
result Transformers with constant depth can achieve low test loss on Markov sequences, matching empirical and theoretical findings.
Transformer model for probabilistic dynamical systems.
problem Modeling high-dimensional dynamical systems from noisy observations.
method Parallel between dynamical systems and language modeling; transformer-based model with geometrical properties; iterative training algorithm.
result Fine-grid approximation of conditional probabilities for high-dimensional systems.
Transformers can simulate MLE for Bayesian network sequences.
problem Understanding transformers' capabilities in Bayesian network sequence generation.
method In-context maximum likelihood estimation (MLE) for autoregressive sequence generation.
result A simple transformer model can estimate Bayesian network probabilities and generate new samples.
New efficient method for inverse Z-transform reduces complexity significantly.
problem Efficient numerical realization of inverse Z-transform for large n.
method Derives sufficient conditions for new scheme, applies to option pricing.
result Significant reduction in complexity for large n, especially for European options.
Study on DiTs' rates of approximation and estimation under various data assumptions.
problem Investigating statistical rates of conditional diffusion transformers.
method Discretization and Taylor expansion of conditional diffusion score function under Hölder smooth data assumption.
result Establishes statistical limits for conditional and unconditional DiTs, offering practical guidance.
Researchers derive an explicit Laplace transform for integrated Volterra Wishart process.
problem Modeling and pricing financial instruments with complex covariance structures.
method Explicit expression for conditional Laplace transform of integrated Volterra Wishart process, linking to matrix Riccati equations.
result Derivation of Laplace transform for a special case of convolution kernel, leading to efficient pricing methods.
A new method tests conditional independence by transforming it into an unconditional problem using transport maps.
problem Testing conditional independence between two random vectors given a third.
method Constructing transport maps to transform conditional independence into unconditional independence, estimating these maps from data using conditional continuous normalizing flow models.
result The proposed method is validated through simulations and real-data analysis, demonstrating practical effectiveness.
For a complete Riemannian metric, a pointwise conformal transformation may lead to a complete or incomplete transformed Riemannian metric, depending on the behavior of the conformal factor. We establish conditions on the growth of the conformal factor towards the infinity of the Riemannian metric, such that the conform…
In this paper we consider Fourier transform techniques to efficiently compute the Value-at-Risk and the Conditional Value-at-Risk of an arbitrary loss random variable, characterized by having a computable generalized characteristic function. We exploit the property of these risk measures of being the solution of an ele…
Improves CRRR for better mobility analysis with DCTM.
problem Unclear interpretation of RRRX parameters.
method Uses DCTM for conditional ranks, cross-fitting, and asymptotic theory.
result Clearer interpretation and improved accuracy in mobility analysis.
Study Lp boundedness of Riesz transform on differential forms for certain manifolds.
problem Investigate Lp-boundedness of the covariant Riesz transform on differential forms. method Analyze Lp-boundedness on weighted Riemannian manifolds under curvature-dimension and lower bound conditions. result Derive Calderón-Zygmund inequality for 1<p≤2 under curvature-dimension condition. Transformation models are a very important tool for applied statisticians and econometricians. In many applications, the dependent variable is transformed so that homogeneity or normal distribution of the error holds. In this paper, we analyze transformation models in a high-dimensional setting, where the set of potent…
Proves conditions for Fourier transforms in rank 1 symmetric spaces.
problem Understanding Fourier transform bounds in symmetric spaces.
method Proves sufficient and necessary conditions using Lipschitz and Fourier type integral conditions.
result Establishes bounds for Fourier transforms in rank 1 symmetric spaces with specific moduli of continuity.
The study uncovers invariant features in healthcare models that traditional methods overlook.
problem Discovering overlooked invariant features in healthcare models.
method Empirical learning of transformations minimizing Wasserstein distance and adding similarity regularization.
result LSTM models and BioBERT reveal invariant features not previously recognized.
Transformer model improves asset allocation by unifying forecasting and optimization.
problem Separation of forecasting and optimization leads to suboptimal portfolios.
method Signature Informed Transformer using path signatures and specialized attention.
result Direct minimization of Conditional Value at Risk improves performance.
A new method uses Transformers for efficient prediction of marked point processes.
problem Efficiently predicting the next event in a sequence given its history.
method Modeling conditional inter-event times with a mixture of log-normals and marks with a Transformer architecture.
result The method achieves state-of-the-art performance and is faster during inference.
MicAugment transfers audio style from few seconds of input to match target conditions.
problem Audio model robustness to diverse acquisition conditions.
method Identifies and applies transformations learned from target audio to input audio.
result MicAugment significantly improves model robustness in downstream tasks.
Paper recovers latent causal structure and linear transformation from indirect observations.
problem Recovering latent causal structure and linear transformation from indirect observations.
method Established sufficient conditions for DAG recovery, leveraged score function properties, and used soft/hard interventions.
result Perfect recovery of latent DAG structure and linear transformation up to scaling using soft interventions, hard interventions with additional hypothesis testing.
We derive precise transformation formulas for synthetic lower Ricci bounds under time change. More precisely, for local Dirichlet forms we study how the curvature-dimension condition in the sense of Bakry-Emery will transform under time change. Similarly, for metric measure spaces we study how the curvature-dimension c…
This paper analyzes deep and wide transformer training dynamics.
problem Understanding the training dynamics of infinitely deep and wide transformers.
method Develops a mean-field framework for gradient-based training of transformers, controlling a neural PDE.
result Establishes a rigorous foundation for gradient-based transformer training, proving convergence to global minima.
Conditions for Penrose-Ward transformation on specific manifolds.
problem Conditions for Penrose-Ward transformation on almost G2-manifolds with almost twistorial structures. method Necessary and sufficient conditions derived through Penrose-Ward transformation.
result Conditions for Penrose-Ward transformation on almost G2-manifolds with almost twistorial structures. This work makes neural sequence models more efficient by controlling computation.
problem Fixed compute for all examples in neural networks.
method Conditional computation to adapt compute to example complexity.
result Conditional Computation Transformer (CCT) improves efficiency and performance.
Enformer and GEnformer use Transformers with stochastic learning to forecast multivariate and spatiotemporal data with uncertainty.
problem Uncertainty quantification in multivariate time series and spatiotemporal forecasting.
method Synthesizing Transformer's expressive power with stochastic learning to model conditional distributions directly.
result Enformer and GEnformer yield calibrated probabilistic forecasts and outperform state-of-the-art baselines.
Investigates the benefits of multi-head attention in Transformers, deriving convergence and generalization guarantees.
problem Underexplored dynamics of multi-head attention in Transformer training and generalization.
method Derives convergence and generalization guarantees for gradient-descent training of a multi-head self-attention model.
result Establishes conditions for initialization that ensure multi-head attention's realizability.
Transformers learn a mesa-optimizer to implement in-context learning.
problem Understanding the convergence of autoregressive training to a mesa-optimizer.
method Investigated a one-layer linear causal self-attention model autoregressively trained by gradient flow.
result Proved that autoregressive training converges to a gradient descent step for an OLS problem, validating the mesa-optimizer hypothesis.
The fundamental task of general density estimation p(x) has been of keen interest to machine learning. In this work, we attempt to systematically characterize methods for density estimation. Broadly speaking, most of the existing methods can be categorized into either using: \textit{a}) autoregressive models to estim…
The study finds conditions for compressing the hidden dimension of Graph Transformers for transductive learning.
problem The challenge of efficiently analyzing and training Graph Transformers for transductive learning.
method Theoretical bounds on hidden dimension compression for Graph Transformers, considering both sparse and dense variants.
result Theoretical findings on how and under what conditions the hidden dimension of Graph Transformers can be compressed.
Transformer-based models overfit financial time series data, leading to increased prediction variance.
problem Forecast collapse of transformer-based models under squared loss in financial time series.
method Theoretical analysis and numerical experiments on high-frequency EUR/USD exchange rate data.
result Increased model expressivity in Transformer-based models leads to spurious fluctuations without reducing bias, resulting in higher prediction variance.
The aim of this article is to provide a systematic analysis of the conditions such that Fourier transform valuation formulas are valid in a general framework; i.e. when the option has an arbitrary payoff function and depends on the path of the asset price process. An interplay between the conditions on the payoff funct…
Injectivity of geodesic ray transform on specific Finsler manifolds proven.
problem Injectivity of geodesic ray transform on spherically symmetric reversible Finsler manifolds.
method Reduction to invertibility of generalized Abel transforms using angular Fourier series and Taylor expansions of geodesics.
result Injectivity of geodesic ray transform proven on specified Finsler manifolds.
Transformer learns CoVaR from financial news, improving systemic risk forecasts.
problem Quantifying systemic financial risk using conditional Value-at-Risk (CoVaR).
method Transformer-based approach integrating financial news articles with market data.
result Transformer CoVaR improves out-of-sample forecasts and identifies market stress periods.
FineMorphs models smooth transformations for multivariate regression.
problem Efficiently modeling complex transformations for multivariate regression.
method Optimal control of affine and diffeomorphic transformations using smooth vector fields.
result FineMorphs can reduce dimensionality and adapt to large datasets.
The paper provides theoretical guarantees for transformation-based models in variational inference.
problem Theoretical justification for transformation-based models in variational inference.
method Theoretical analysis of non-linear latent variable models and Gaussian process priors.
result Theoretical guarantees for implicit variational inference, achieving optimal risk bounds and approximating the true posterior.
Paper analyzes risk bounds for in-context learning in multiclass classification.
problem Risk bounds for in-context learning in multiclass classification.
method Formalizes tasks as sequences of labeled examples and queries, estimates conditional class probabilities, establishes oracle inequality for KL divergence.
result ICL achieves minimax optimal rate for conditional probability estimation.
This work proposes an efficient autoregressive model for text generation.
problem The challenge of generating high-quality text with autoregressive models.
method Introduces a cascaded decoding approach using Markov transformers to achieve sub-linear parallel time generation.
result Shows competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
Extends Gaussian process regression for non-Gaussian data.
problem Inadequate modeling of uncertainty and over-smoothing in non-Gaussian datasets.
method Time-changed Gaussian processes with Lévy processes.
result Improved modeling of heavy-tailed non-Gaussian behaviors.
trVAE improves conditional out-of-sample generation for unpaired data.
problem Challenges in generating high-dimensional samples conditional on low-dimensional descriptors out-of-sample.
method trVAE uses maximum mean discrepancy (MMD) to match distributions across conditions in the decoder layer.
result Improved robustness and accuracy in predicting cellular perturbation responses and disease.
New analysis shows RPE-based Transformers can't approximate all functions.
problem Understanding the limitations of RPE-based Transformers in approximating continuous functions.
method Mathematical analysis and development of a novel attention module (URPE) to overcome limitations.
result RPE-based Transformers can't approximate all continuous sequence-to-sequence functions, even with depth and width.
Enhances neural processes to learn from multiple related datasets.
problem Improving predictions from datasets with shared similarities.
method Developed the in-context in-context learning pseudo-token TNP (ICICL-TNP) to condition on both sets of datapoints and sets of datasets.
result Demonstrated the importance and effectiveness of in-context in-context learning.
We construct a local action of the group of rational maps from S2 to GL(n,C) on local solutions of flows of the ZS-AKNS sl(n,C)-hierarchy. We show that the actions of simple elements (linear fractional transformations) give local Bäcklund transformations, and we derive a permutability formula from different fact…
We tackle causal inference under conditional moment restrictions using importance weighting.
problem Challenges in causal inference under conditional moment restrictions, especially in high-dimensional settings.
method Transform conditional moment restrictions to unconditional moment restrictions through importance weighting.
result Successfully estimate nonparametric functions defined under conditional moment restrictions.