Recent research has used margin theory to analyze the generalization performance for deep neural networks (DNNs). The existed results are almost based on the spectrally-normalized minimum margin. However, optimizing the minimum margin ignores a mass of information about the entire margin distribution, which is crucial …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Marginal Distributions”
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased" choice corresponds to the distribution of maximum entropy. The calculation of t…
Support vector regression (SVR) is one of the most popular machine learning algorithms aiming to generate the optimal regression curve through maximizing the minimal margin of selected training samples, i.e., support vectors. Recent researchers reveal that maximizing the margin distribution of whole training dataset ra…
A framework estimates categorical distributions under constraints, ensuring generality and uniqueness.
problem Estimating categorical distributions summarizing sample data under marginal constraints.
method Theoretical framework + Iterative Proportional Fitting (IPF) to estimate the distribution.
result A unique categorical distribution of Maximum Entropy under marginal constraints exists and is estimated.
Margin enlargement over training data has been an important strategy since perceptrons in machine learning for the purpose of boosting the robustness of classifiers toward a good generalization ability. Yet Breiman (1999) showed a dilemma that a uniform improvement on margin distribution does NOT necessarily reduces ge…
In the absence of prior knowledge, ordinal embedding methods obtain new representation for items in a low-dimensional Euclidean space via a set of quadruple-wise comparisons. These ordinal comparisons often come from human annotators, and sufficient comparisons induce the success of classical approaches. However, colle…
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on…
In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from to , while - the margin r…
Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predi…
Study finds a method to discover causal relationships that are invariant to marginal distributions.
problem Current causal discovery methods are sensitive to marginal distributions, leading to unreliable results.
method Proposes a non-parametric estimator that marginalizes the marginals to find intrinsic causal relationships.
result The proposed method yields causal estimators competitive with current methodologies and emphasizes uncertainty.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L_2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the s…
Paper reinterprets marginal productivity theory using vectorial products, challenging traditional ethical interpretations.
problem Challenges traditional ethical interpretations of marginal productivity theory.
method Formulates marginal productivity theory using vectorial marginal products, contrasting with traditional scalar approach.
result Vectorial marginal products conflict with traditional distributive shares picture of property.
New margin-based learning guarantees improve generalization bounds.
problem Improving generalization bounds for machine learning models.
method Relative deviation margin bounds using empirical margin loss and Rademacher complexity.
result Distribution-dependent generalization bounds for unbounded loss functions.
Gradient-based methods can be biased by distributional asymmetries in bivariate categorical data.
problem Gradient-based causal discovery methods can be biased by distributional asymmetries in bivariate categorical data.
method Identified and examined two distributional biases: Marginal Distribution Asymmetry and Marginal Distribution Shift Asymmetry. Employed two simple models to demonstrate and control these biases.
result Gradient-based methods can be biased by distributional asymmetries, and these biases can be controlled.
Variational Prediction simplifies Bayesian inference without test time costs.
problem Bayesian inference's computational costs and posterior predictive distribution marginalization.
method Variational Prediction learns a variational approximation to the posterior predictive distribution using a variational bound.
result Directly learns a variational approximation to the posterior predictive distribution without test time marginalization costs.
In this work, we study a new approach to optimizing the margin distribution realized by binary classifiers. The classical approach to this problem is simply maximization of the expected margin, while more recent proposals consider simultaneous variance control and proxy objectives based on robust location estimates, in…
New insights into deep learning: reducing training data significantly improves performance.
problem Understanding and improving generalization in deep learning models.
method Analyzing the distribution of classification margins and dynamically reducing the training set.
result The area under the curve of the margin distribution is a good measure of generalization.
New findings on PAC learning and marginal distribution estimation.
problem Understanding how PAC learning relates to marginal distribution estimation under distributional constraints.
method Revisited the connection between PAC learning, uniform convergence, and density estimation, considering a known family of marginal distributions.
result PAC learning is sandwiched between two refined models of density estimation, differing only in whether the learner knows the set of well-estimated events in H.
Novel proof shows continuity of optimal transport feasible set mapping.
problem Continuity of feasible set mapping in optimal transport problems.
method Presented a novel and shorter proof of continuity.
result Established continuity of the feasible set mapping.
Estimates high-dimensional posterior densities by marginal distributions and neural networks.
problem High-dimensional probability density estimation for inference is difficult.
method Direct estimation of lower-dimensional marginal distributions, using Moment Networks for fast computation of moments.
result Demonstrates estimation of gravitational wave time series and applications in cosmology.
On the Insufficiency of the Large Margins Theory in Explaining the Performance of Ensemble Methodsstat.ML
Boosting and other ensemble methods combine a large number of weak classifiers through weighted voting to produce stronger predictive models. To explain the successful performance of boosting algorithms, Schapire et al. (1998) showed that AdaBoost is especially effective at increasing the margins of the training data. …
As shown in recent research, deep neural networks can perfectly fit randomly labeled data, but with very poor accuracy on held out data. This phenomenon indicates that loss functions such as cross-entropy are not a reliable indicator of generalization. This leads to the crucial question of how generalization gap should…
AIS uses a suboptimal extended target distribution, which this paper improves using SGM.
problem Improving the efficiency of Annealed Importance Sampling for marginal likelihood estimation.
method Leveraging score-based generative modeling to approximate the optimal extended target distribution.
result Demonstrated novel, differentiable AIS procedures on synthetic and real-world data.
New method controls error in low-dimensional marginals of spatial models.
problem Inaccurate approximation of low-dimensional marginals in spatial models.
method Stein's method with δ-locality condition for spatial models.
result Uniform error bound for marginals of approximate distributions.
Sequential Monte Carlo techniques are useful for state estimation in non-linear, non-Gaussian dynamic models. These methods allow us to approximate the joint posterior distribution using sequential importance sampling. In this framework, the dimension of the target distribution grows with each time step, thus it is nec…
Dis-entangling Mixture of Interventions on a Causal Bayesian Network Using Aggregate Observationscs.LG
We study the problem of separating a mixture of distributions, all of which come from interventions on a known causal bayesian network. Given oracle access to marginals of all distributions resulting from interventions on the network, and estimates of marginals from the mixture distribution, we want to recover the mixi…
The paper examines risk aggregation under mixtures of marginals, finding that more homogeneous distributions lead to larger uncertainty.
problem Investigating the impact of mixing on risk aggregation uncertainty.
method Analyzes ordering relations and inequalities for aggregation sets under distribution and quantile mixtures.
result More homogeneous marginals result in larger aggregation sets, indicating greater model uncertainty.
Proposes MFSWB for marginal fairness in SWB, improving efficiency and performance.
problem Achieving marginal fairness in SWB averaging.
method Defining MFSWB as a constrained SWB problem, proposing two surrogate problems and a new slicing distribution.
result Surrogate MFSWB problems effectively minimize distances to marginals and encourage marginal fairness.
Near-Optimal SQ Lower Bounds for Agnostically Learning Halfspaces and ReLUs under Gaussian Marginalscs.LG
The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.
problem Agnostically learning halfspaces and ReLUs under Gaussian marginals.
method Statistical Query (SQ) lower bounds analysis.
result Proves SQ lower bounds of for both problems.
Efficient algorithm for Bayesian networks reduces marginal probability distribution computation.
problem Exact computation of marginal probability distribution is NP-hard for categorical variables in Bayesian networks.
method Divide-and-conquer approach exploiting graphical properties of Bayesian networks.
result Novel algorithm outperforms state-of-the-art methods in classification and cancer subtype identification.
Maximum entropy distributions with discrete support in dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions have been extensively studied, basic questions such as: Do max-entropy distributio…
We extend probabilistic programming to handle conditioning on marginal distributions.
problem Conditioning probabilistic programs on marginal distributions of observable variables.
method We define and implement stochastic conditioning, allowing inference in probabilistic programs conditioned on marginal distributions.
result We demonstrate the effectiveness of stochastic conditioning in various real-life scenarios.
COMET Flows model multivariate extremes with heavy tails and asymmetric dependence.
problem Normalizing flows struggle with multivariate extremes and asymmetric tail dependence.
method COMET Flows decomposes modeling into marginal and copula parts; uses tail belief and kernel density for marginals, and low-dimensional manifold for tail dependence.
result COMET Flows outperform other models in capturing heavy-tailed marginals and asymmetric tail dependence.
Identification and Bounding of Central Moments of Causal Effects Using Marginal Moments Informationstat.ME
This paper identifies and bounds ICE central moments using PO marginal central moments.
problem Identifying and characterizing treatment effect heterogeneity.
method Using only marginal central moments of potential outcomes, the paper identifies and bounds central moments of individual causal effects.
result Identification and bounding of central moments of ICE using marginal moments of POs.
Proposes a method to construct risk-neutral marginals from arbitrage-free option prices.
problem Lack of risk-neutral marginals that are free of arbitrage and easy to use.
method Explicit construction of risk-neutral marginals from discrete arbitrage-free option prices.
result Explicit construction guarantees risk-neutral marginals free of butterfly and calendar arbitrage.
New method calibrates local volatility models to marginal distributions.
problem Calibrating local volatility models to specific marginal distributions.
method Inspired by volatility interpolation, constructs time-homogeneous or continuous local volatility functions.
result Efficient numerical algorithms for constructing local volatility functions.
Bayesian inference with finitely wide neural networkscond-mat.dis-nn
Bayesian inference for wide neural networks using Edgeworth expansion.
problem Analyzing the non-Gaussian behavior of wide neural networks in Bayesian inference.
method Proposed a non-Gaussian distribution using multivariate Edgeworth expansion for finite-width neural networks.
result Derived non-Gaussian posterior distribution in Bayesian regression tasks.
This paper studies convergence properties of multivariate distributions constructed by endowing empirical margins with a copula. This setting includes Latin Hypercube Sampling with dependence, also known as the Iman--Conover method. The primary question addressed here is the convergence of the component sum, which is r…
A new approach for instance-optimal learning that bypasses impossibility results.
problem Impossibility of achieving marginal-by-marginal guarantees for all marginals.
method Introduces relatively smart learning, which requires competition only with certifiable semi-supervised guarantees.
result One-Inclusion Graph learner is relatively smart up to squaring the sample complexity.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
Optimizes AIS hyperparameters for efficient marginal likelihood estimation.
problem Limited computation budget affects AIS performance.
method Flexible intermediary distributions defined by residual density, parameter sharing, and fix linear schedule.
result Optimized-Path AIS reduces sampling iterations and improves performance.
Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimationstat.ML
We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a sequential factorization of the marginal likelihood in such settings by using stochastic gradient Markov Chain Monte Carlo techniques. This appro…
Optimizes risk measures given known marginal distributions of two unknown factors.
problem Determining an upper bound for spectral risk measures with unknown joint distribution.
method Introduces Maximum Spectral Measure (MSP) as a worst-case risk measure, formulated as an optimization problem with a more general objective function.
result Characterizes the continuity properties of the optimal value function and optimal solution set with respect to marginal distributions.
Paper connects rejection learning to Bhattacharyya divergence.
problem Learning models to abstain from predictions.
method Developed a link between rejection and thresholding different statistical divergences, focusing on Bhattacharyya divergence.
result Rejector obtained by joint ideal distribution corresponds to thresholding of skewed Bhattacharyya divergence.
Standard methods for anomaly detection assume that all features are observed at both learning time and prediction time. Such methods cannot process data containing missing values. This paper studies five strategies for handling missing values in test queries: (a) mean imputation, (b) MAP imputation, (c) reduction (redu…
Max-margin learning is a powerful approach to building classifiers and structured output predictors. Recent work on max-margin supervised topic models has successfully integrated it with Bayesian topic models to discover discriminative latent semantic structures and make accurate predictions for unseen testing data. Ho…
New method for efficient marginalization of discrete latent variables in neural networks.
problem Computational challenges in training models with discrete latent variables.
method Parameterizing discrete distributions using sparse mappings (sparsemax and structured variants) to reduce support and enable efficient marginalization.
result Achieved good performance in various tasks with efficient and practical training.
Study bounds financial path expectations using martingale distributions.
problem Bounding path-dependent financial expectations over martingale distributions.
method Relaxed martingale optimal transport problem, approximated via linear programming.
result Empirical relaxation can be approximated within O(n^(-1/2)) error.