ICA reveals deep learning's feature learning mechanisms from non-Gaussian data.
problem Understanding feature learning from non-Gaussian inputs in deep neural networks.
method Investigates ICA and SGD on synthetic and real data.
result FastICA requires n≳d4 samples for single non-Gaussian direction recovery, while SGD outperforms and optimised SGD reaches n≳d2. The ACCRU framework improves probabilistic forecasts by capturing input-dependent uncertainty.
problem Uncertainty in deterministic predictions, especially for skewed and non-Gaussian errors.
method Neural network trained with a loss function balancing accuracy and reliability to learn input-dependent, non-Gaussian uncertainty distributions.
result Improves probabilistic forecasts relative to existing methods, capturing skewed and non-Gaussian errors.
GGMPs improve non-Gaussian conditional density estimation.
problem Multimodality, heteroscedasticity, and strong non-Gaussianity in conditional density estimation.
method GGMP combines local Gaussian mixture fitting, cross-input component alignment, and per-component heteroscedastic GP training.
result GGMPs improve distributional approximation on synthetic and real-world datasets.
Random neural networks with ReLU activations are non-Gaussian processes.
problem Understanding the behavior of neural networks with random initialization and rectified linear units.
method Proving these networks are non-Gaussian processes and deriving their properties.
result These networks can converge to non-Gaussian processes under certain conditions.
Robust method estimates state, input, and parameters of linear systems online.
problem Joint estimation of state, input, and parameters in noisy or outlier-prone measurements.
method Combines recursive, alternating, and iteratively-reweighted least squares into a single algorithm.
result Good performance in presence of outliers and compared to state-of-the-art methods.
Extends Gaussian process regression for non-Gaussian data.
problem Inadequate modeling of uncertainty and over-smoothing in non-Gaussian datasets.
method Time-changed Gaussian processes with Lévy processes.
result Improved modeling of heavy-tailed non-Gaussian behaviors.
Gaussian Process (GP) regression models typically assume that residuals are Gaussian and have the same variance for all observations. However, applications with input-dependent noise (heteroscedastic residuals) frequently arise in practice, as do applications in which the residuals do not have a Gaussian distribution. …
Learning rate needs to decrease with higher data moments for effective ICA in high dimensions.
problem Slower convergence of ICA in high-dimensional data with high-order moments.
method High-dimensional ODE analysis of ICA algorithm under controlled moment structure.
result Critical learning rate threshold for effective ICA when moments are high.
The discovery of non-linear causal relationship under additive non-Gaussian noise models has attracted considerable attention recently because of their high flexibility. In this paper, we propose a novel causal inference algorithm called least-squares independence regression (LSIR). LSIR learns the additive noise model…
This paper improves GP for learning complex data distributions.
problem Vanilla Gaussian processes struggle with complex data distributions.
method Introduces scalable GP paradigms with latent variables and variational inference.
result Scalable modulated GPs, especially latent GPs, learn diverse data distributions better.
We analyze the convergence of (stochastic) gradient descent algorithm for learning a convolutional filter with Rectified Linear Unit (ReLU) activation function. Our analysis does not rely on any specific form of the input distribution and our proofs only use the definition of ReLU, in contrast with previous works that …
The Kalman filter is extensively used for state estimation for linear systems under Gaussian noise. When non-Gaussian Lévy noise is present, the conventional Kalman filter may fail to be effective due to the fact that the non-Gaussian Lévy noise may have infinite variance. A modified Kalman filter for linear systems wi…
The statistical dependencies which independent component analysis (ICA) cannot remove often provide rich information beyond the linear independent components. It would thus be very useful to estimate the dependency structure from data. While such models have been proposed, they usually concentrated on higher-order corr…
New framework models neural systems with random architecture on manifolds.
problem Complex, uncertain systems with non-Gaussian outputs.
method Latent random field on compact manifold generates neural architecture and weights.
result Synthetic neural systems can produce stochastic outputs for deterministic inputs.
This work extracts stochastic dynamical systems with α-stable Lévy noise.
problem Extracting data-driven governing laws of dynamical systems with non-Gaussian noise.
method End-to-end deep learning approach for learning drift and diffusion coefficients for α-stable Lévy noise. result Effectiveness of the method confirmed by numerical experiments.
The paper analyzes the non-Gaussian behavior of inflation and unemployment over 70 years using multifractal methods.
problem Capturing unusual fluctuations in inflation and unemployment over long periods.
method Coupled multifractal approach to analyze non-Gaussian distributions of inflation and unemployment over 70 years.
result The non-Gaussianity of unemployment is noticeable only for periods smaller than 1 year, while inflation's non-Gaussianity persists across all time scales.
Researchers derive exact priors for finite Bayesian neural networks.
problem Understanding non-Gaussian priors in finite Bayesian neural networks.
method Analytical derivation of function space priors for finite fully-connected feedforward networks.
result Exact solutions for priors of finite networks, including Meijer G-function for linear networks and mixtures for ReLU networks.
Independent component analysis (ICA) decomposes multivariate data into mutually independent components (ICs). The ICA model is subject to a constraint that at most one of these components is Gaussian, which is required for model identifiability. Linear non-Gaussian component analysis (LNGCA) generalizes the ICA model t…
Conditional Density Estimation (CDE) models deal with estimating conditional distributions. The conditions imposed on the distribution are the inputs of the model. CDE is a challenging task as there is a fundamental trade-off between model complexity, representational capacity and overfitting. In this work, we propose …
Paper uses optimal transport for Bayesian filtering, deriving new EnKF and FPF formulations.
problem Bayesian filtering for nonlinear systems with non-Gaussian observations.
method Optimal transport theory applied to Bayes' law, constructing Brenier maps.
result New variational formulations of EnKF and FPF for non-Gaussian settings.
Paper proposes a generalized precision matrix for t-Student distributions to improve portfolio optimization.
problem Limitations of inverse covariance matrix in non-Gaussian settings.
method Exploits local dependence function to define generalized precision matrix (GPM) for multivariate t-Student distribution.
result GPM leads to statistically significant lower out-of-sample variances in minimum-variance portfolios.
Improves graph-based active learning for non-Gaussian models.
problem Efficiently selecting data points for labeling in graph-based semi-supervised learning.
method Approximates non-Gaussian distributions, introduces rank-one update and model change acquisition function.
result Enhanced active learning for graph-based SSL under non-Gaussian models.
This research develops an evolutionary approach to discover non-Gaussian stochastic dynamical systems.
problem Discovering explicit governing equations of stochastic dynamical systems with Lévy noise from data.
method ESSR approach using genetic programming, sparse regression, and nonlocal Kramers-Moyal formulas.
result The approach effectively extracts non-Gaussian stochastic dynamical systems from sample path data.
The problem of Non-Gaussian Component Analysis (NGCA) is about finding a maximal low-dimensional subspace E in Rn so that data points projected onto E follow a non-gaussian distribution. Although this is an appropriate model for some real world data analysis problems, there has been little progress on t…
New algorithm for learning causal structures with disjoint cycles in linear non-Gaussian models.
problem Learning causal structures with cycles in linear non-Gaussian models.
method Characterizing when graphs determine the same model, using quadratic and cubic polynomial relations, and a strategy of decorrelating cycles and multivariate regression.
result Consistent and computationally efficient algorithm for learning causal structures with disjoint cycles.
The paper finds non-Gaussian directions in high-dimensional data using Wasserstein distance.
problem Locating interesting non-Gaussian features in high-dimensional data.
method Projection pursuit using 2-Wasserstein distance to maximize the difference from Gaussian.
result Statistical guarantees for accurately approximating an unknown low-dimensional non-Gaussian subspace.
Proposes efficient Gaussian approximations for non-Gaussian likelihoods.
problem Computational challenges in learning and inference with non-Gaussian likelihoods.
method Variational inference and moment matching in transformed bases.
result Good approximation quality for binary and multiclass classification.
A new vine copula mixture model improves clustering accuracy for non-Gaussian data.
problem Finite mixture models struggle with asymmetric tail dependencies and non-elliptical clusters.
method Proposes a vine copula mixture model for clustering non-Gaussian data, addressing model selection and parameter estimation.
result Significant improvement in clustering accuracy for data with asymmetric tail dependencies or non-Gaussian margins.
Sparse non-Gaussian component analysis (SNGCA) is an unsupervised method of extracting a linear structure from a high dimensional data based on estimating a low-dimensional non-Gaussian data component. In this paper we discuss a new approach to direct estimation of the projector on the target space based on semidefinit…
This paper presents a method for efficient density estimation in nonlinear systems.
problem Accurate representation of non-Gaussian distributions in nonlinear dynamical systems is challenging.
method Uses Seminonparametric (SNP) densities with probabilists' Hermite polynomial basis and Monte Carlo approximation for maximum likelihood estimation.
result Demonstrates that the method can accurately capture non-Gaussian density structure and compute quantiles using fewer samples than raw Monte Carlo.
Paper solves NGCA for discrete distributions using LLL method.
problem Learning hidden non-Gaussian components in discrete distributions.
method Utilizes LLL lattice basis reduction method.
result Sample and computationally efficient algorithm for NGCA in discrete distributions.
Wavelet scattering spectra model non-Gaussian time-series, proving scale invariance for self-similar processes.
problem Modeling non-Gaussian time-series with stationary increments.
method Complex wavelet transform for scale variations, joint correlation matrix for scale dependencies, second wavelet transform for diagonalization, maximum entropy models conditioned by scattering spectra coefficients.
result Scattering spectra of self-similar processes are scale invariant, allowing statistical testing and generation of new time-series.
Algorithm learns non-Gaussian graphical models via Hessian scores and triangular transport.
problem Learning graph structure from non-Gaussian data.
method Score based on integrated Hessian information, coupled with triangular transport map.
result Algorithm successfully recovers graph structure for non-Gaussian data.
The paper examines non-Gaussian models for financial data.
problem Modeling financial data with non-Gaussian distributions.
method Analysis of multivariate non-Gaussian models focusing on parsimony, dependence structure, and computational aspects.
result Characterization and calibration of models for financial log-returns.
Max-margin classifiers' behavior is studied in high dimensions with non-Gaussian features.
problem Understanding the role of featurization maps and high-dimensional misclassification error.
method High-dimensional asymptotics, Gaussian model, support vector representation.
result Asymptotic behavior of max-margin classifiers is determined by feature covariance and label covariance.
Non-Gaussian component analysis (NGCA) is an unsupervised linear dimension reduction method that extracts low-dimensional non-Gaussian "signals" from high-dimensional data contaminated with Gaussian noise. NGCA can be regarded as a generalization of projection pursuit (PP) and independent component analysis (ICA) to mu…
This work extends Tweedie's formulae to non-Gaussian processes for better diffusion model generation.
problem Limited exploration of non-Gaussian diffusion models and corresponding Tweedie's formulae.
method Extended Tweedie's formulae to geometric Brownian motion, squared Bessel, and Cox-Ingersoll-Ross processes.
result Demonstrated potential of non-Gaussian models in image and financial time series generation.
Study large deviations in fractional volatility models with non-Gaussian volatility.
problem Large deviations in fractional volatility models with non-Gaussian volatility.
method Established a small-noise large deviation principle for log-price.
result Logarithmic call price asymptotics for large strikes in a special case.
Transformer with denoising diffusion improves probabilistic density estimation.
problem Estimating non-Gaussian and multimodal probability distributions for regression problems.
method Training a denoising diffusion head on top of a Transformer model.
result The model provides reasonable probability density estimation for high-dimensional inputs.
Gaussian processes are ubiquitous in nature and engineering. A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes. Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors. The methodo…
New method certifies anti-concentration for various non-Gaussian distributions.
problem Efficiently certifying anti-concentration for non-Gaussian distributions.
method Sum-of-Squares relaxation of integer program for anti-concentration.
result Quasi-polynomial time certificates for non-Gaussian distributions.
New bounds for kernel regression under non-Gaussian noise.
problem Uncertainty quantification for function estimates from noisy observations.
method Novel non-asymptotic probabilistic uniform error bounds for kernel-based regression.
result Proposed bounds apply to a broad class of non-Gaussian noise distributions.
Study of a risk-averse informed trader in a multi-asset market with non-Gaussian prices.
problem Existence of equilibrium in a multi-asset market with non-Gaussian prices and a risk-averse informed trader.
method Constructed equilibrium using Fokker-Planck equation and coupled partial differential equations with an optimal transport constraint.
result Equilibrium exists in a market with multiple assets and non-Gaussian prices.
This paper tackles the challenge presented by small-data to the task of Bayesian inference. A novel methodology, based on manifold learning and manifold sampling, is proposed for solving this computational statistics problem under the following assumptions: 1) neither the prior model nor the likelihood function are Gau…
Improved Kalman filter for non-linear, non-Gaussian data.
problem Estimating hidden variables with non-linear, non-Gaussian observations.
method Reproduces and extends Burkhart et al.'s discriminative Kalman filter.
result Enhanced filter performance for complex observation models.
Time series models generalize ARMA and ARFIMA with non-Gaussian dependence.
problem Modeling non-Gaussian serial dependence in time series data.
method Infinite-order partial copula dependence in s-vine processes.
result Rich class of models that generalize linear processes.
Method extracts governing laws from non-Gaussian stochastic systems data.
problem Modeling complex dynamics with non-Gaussian Lévy noise.
method Data-driven method to extract stochastic dynamical systems from noisy data.
result Established a theoretical framework and numerical algorithm to compute Lévy jump measure, drift, and diffusion.
Efficiently learns linear non-Gaussian DAGs with noisy nodes.
problem Learning DAGs with non-Gaussian noise and diverging number of nodes.
method Proposes a novel method using topological layers for bottom-up reconstruction and consistent parent-child relations.
result Topological layers can be exactly reconstructed and parent-child relations established without faithfulness assumption.