The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased" choice corresponds to the distribution of maximum entropy. The calculation of t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “maximum margin solution”
Study connects spectral clustering to maximum margin and level set estimation.
problem Connecting spectral clustering to maximum margin and level set estimation.
method Obtained bounds on eigenvectors of graph Laplacian matrices in terms of cluster separation and connectivity. Showed sensitivity mitigation by removing outliers and estimating level sets.
result Spectral clustering converges to maximum margin clustering as scaling parameter approaches zero.
The paper presents a method to estimate joint interventional distributions from marginal interventional data.
problem Estimating joint interventional distributions from marginal interventional data.
method The paper extends the Causal Maximum Entropy method to use interventional data and employs Lagrange duality to prove the solution lies in the exponential family.
result The method allows for causal feature selection and inference of joint interventional distributions.
Mirror flow optimizes separable data problems, converging to a maximum margin classifier.
problem Optimizing classification problems with separable data using mirror flow.
method Examine mirror flow on linearly separable classification problems, focusing on the horizon function of the mirror potential.
result Mirror flow converges to a maximum margin classifier for separable data under certain conditions.
We obtain bounds on the distribution of the maximum of a martingale with fixed marginals at finitely many intermediate times. The bounds are sharp and attained by a solution to -marginal Skorokhod embedding problem in Obłój and Spoida [An iterated Azéma-Yor type embedding for finitely many marginals (2013) Preprint]…
Optimizes risk measures given known marginal distributions of two unknown factors.
problem Determining an upper bound for spectral risk measures with unknown joint distribution.
method Introduces Maximum Spectral Measure (MSP) as a worst-case risk measure, formulated as an optimization problem with a more general objective function.
result Characterizes the continuity properties of the optimal value function and optimal solution set with respect to marginal distributions.
We solve the -marginal Skorokhod embedding problem for a continuous local martingale and a sequence of probability measures which are in convex order and satisfy an additional technical assumption. Our construction is explicit and is a multiple marginal generalisation of the Azema and Yor (1979) soluti…
We consider the problem of learning Bayesian network classifiers that maximize the marginover a set of classification variables. We find that this problem is harder for Bayesian networks than for undirected graphical models like maximum margin Markov networks. The main difficulty is that the parameters in a Bayesian ne…
New research shows the maximum ℓ1-margin classifier doesn't adapt to sparse ground truths.
problem Understanding the limitations of the maximum ℓ1-margin classifier in high-dimensional settings.
method Analyzing convergence and prediction error rates of the maximum ℓ1-margin classifier.
result Proves tight upper and lower bounds for prediction error, showing benign overfitting.
We introduce a useful tool for analyzing boosting algorithms called the ``smooth margin function,'' a differentiable approximation of the usual margin for boosting algorithms. We present two boosting algorithms based on this smooth margin, ``coordinate ascent boosting'' and ``approximate coordinate ascent boosting,'' w…
The paper analyzes the maximum margin algorithm's performance on noisy data.
problem Analyzing the performance of maximum margin algorithm on noisy data.
method Finite-sample analysis of maximum margin algorithm applied to noisy data.
result The maximum margin algorithm can achieve nearly optimal population risk with sufficient over-parameterization.
Maximum entropy distributions with discrete support in dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions have been extensively studied, basic questions such as: Do max-entropy distributio…
We give polynomial-time algorithms for the exact computation of lowest-energy (ground) states, worst margin violators, log partition functions, and marginal edge probabilities in certain binary undirected graphical models. Our approach provides an interesting alternative to the well-known graph cut paradigm in that it …
In many real-world applications, data is not collected as one batch, but sequentially over time, and often it is not possible or desirable to wait until the data is completely gathered before analyzing it. Thus, we propose a framework to sequentially update a maximum margin classifier by taking advantage of the Maximum…
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
New insights into optimization and generalization for linear models.
problem Understanding the implicit regularization of optimization methods for linear models.
method Investigating the norms minimized by interpolating solutions and using projections to move between solutions.
result Proving that for over-parameterized linear classification, projections onto the data-span enable the use of under-parameterized techniques.
Study shows how over-parameterized classifiers can still perform well on noisy data.
problem Understanding how maximum margin classifiers perform in over-parameterized settings with noisy data.
method Analyzes maximum margin classifiers on sub-Gaussian mixtures, providing risk bounds.
result Characterizes conditions for 'benign overfitting' in linear classification problems.
New algorithm reduces variance in Monte Carlo simulations using deep neural networks and policy gradients.
problem Reducing variance in Monte Carlo simulations for estimating function values.
method Optimal correlation search using deep neural networks and policy gradients.
result Optimal correlation function reduces variance by approximating and calibrating policy.
Gradient descent aligns weights in deep linear networks for binary classification.
problem Aligning weights in deep linear networks for binary classification.
method Gradient descent applied to strictly decreasing loss functions.
result Normalized weight matrices align across layers, converging to the maximum margin solution.
We consider the problem of classifying data manifolds where each manifold represents invariances that are parameterized by continuous degrees of freedom. Conventional data augmentation methods rely upon sampling large numbers of training examples from these manifolds; instead, we propose an iterative algorithm called M…
A framework estimates categorical distributions under constraints, ensuring generality and uniqueness.
problem Estimating categorical distributions summarizing sample data under marginal constraints.
method Theoretical framework + Iterative Proportional Fitting (IPF) to estimate the distribution.
result A unique categorical distribution of Maximum Entropy under marginal constraints exists and is estimated.
The marginal maximum a posteriori probability (MAP) estimation problem, which calculates the mode of the marginal posterior distribution of a subset of variables with the remaining variables marginalized, is an important inference problem in many models, such as those with hidden variables or uncertain parameters. Unfo…
This work analyzes the maximum-margin bias in quasi-homogeneous neural networks.
problem Analyzing the maximum-margin bias in quasi-homogeneous neural networks.
method Geometric analysis of gradient dynamics for quasi-homogeneous models.
result Gradient flow implicitly favors a subset of parameters, leading to asymmetric norm minimization.
A new method improves maximum margin criterion for better pattern analysis.
problem Handling high dimensionality and large data in pattern analysis.
method Introducing an improved maximum margin criterion (MMC) and its variants.
result Experimental results show the MMC methods are effective in complex scenarios.
Unified approach for optimizing predictions in linear programming and inverse problems.
problem Optimizing predictions in linear programming and inverse problems.
method Maximum optimality margin approach.
result Unified approach that balances computational efficiency and theoretical properties.
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-li…
Graphical models with bi-directed edges (<->) represent marginal independence: the absence of an edge between two vertices indicates that the corresponding variables are marginally independent. In this paper, we consider maximum likelihood estimation in the case of continuous variables with a Gaussian joint distributio…
Learning a regression function using censored or interval-valued output data is an important problem in fields such as genomics and medicine. The goal is to learn a real-valued prediction function, and the training output labels indicate an interval of possible values. Whereas most existing algorithms for this task are…
Exact partial information decompositions for Gaussian systems based on dependency constraintscond-mat.stat-mech
The Partial Information Decomposition (PID) [arXiv:1004.2515] provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method (Idep) has recently been proposed for computing a two-predictor PID over discrete spaces. [arXiv:1709.06653] A lattice of maximum en…
Supervised topic models utilize document's side information for discovering predictive low dimensional representations of documents. Existing models apply the likelihood-based estimation. In this paper, we present a general framework of max-margin supervised topic models for both continuous and categorical response var…
Gradient descent dynamics in deep networks leads to optimal margin solutions.
problem Controlling the complexity of deep networks for generalization.
method Gradient descent dynamics on normalized weights.
result Gradient descent dynamics converge to optimal margin solutions.
A fast method for training linear classifiers maximizes margins.
problem Training linear classifiers with maximum margins.
method Momentum-based gradient method derived from convex dual with Nesterov acceleration.
result Exponentially faster convergence rate compared to standard methods.
New algorithm improves latent variable model estimation.
problem Estimating parameters in latent variable models.
method Jarzynski-adjusted Langevin algorithm (JALA) for SMC methods.
result JALA-EM provides maximum marginal likelihood estimate.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
New method resolves nonidentifiability in mixture models.
problem Nonidentifiability in marginal models of mixture models.
method Introducing an effective temperature to generalize the marginal likelihood.
result Maximization of the generalized likelihood leads to unique results.
We provide a detailed study on the implicit bias of gradient descent when optimizing loss functions with strictly monotone tails, such as the logistic loss, over separable datasets. We look at two basic questions: (a) what are the conditions on the tail of the loss function under which gradient descent converges in the…
Paper analyzes GMM for separable data with various parameter structures.
problem Classifying separable data with logistic models and their generalizations.
method Introduces and analyzes Generalized Margin Maximizer (GMM) for logistic models with specific parameter structures.
result GMM outperforms max-margin classifiers in various parameter settings and structures.
Recently, there has been much interest in finding globally optimal Bayesian network structures. These techniques were developed for generative scores and can not be directly extended to discriminative scores, as desired for classification. In this paper, we propose an exact method for finding network structures maximiz…
Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.
problem Characterizing the implicit bias of gradient descent on linearly separable data.
method Primal-dual analysis with smoothed margin for general losses, and exponential loss with specific step sizes.
result Proves faster convergence rates for implicit bias and margin maximization.
Novel proof shows continuity of optimal transport feasible set mapping.
problem Continuity of feasible set mapping in optimal transport problems.
method Presented a novel and shorter proof of continuity.
result Established continuity of the feasible set mapping.
Deep neural networks can generalize well even with perfect fits to noisy data.
problem Understanding the conditions under which deep neural networks generalize well in the presence of noise.
method Comprehensive study of linear maximum margin classifiers, focusing on noisy and noiseless cases.
result Discovery of a phase transition in test error bounds for the noisy model.
General Vector Machinestat.ML
The support vector machine (SVM) is an important class of learning machines for function approach, pattern recognition, and time-serious prediction, etc. It maps samples into the feature space by so-called support vectors of selected samples, and then feature vectors are separated by maximum margin hyperplane. The pres…
Weisfeiler-Leman struggles with graph isomorphism; enhanced architectures improve generalization.
problem Graph isomorphism problem and limited expressivity of -WL.
method Augmenting -WL and MPNNs with subgraph information, employing margin theory, and introducing provable generalization kernels.
result Increased expressivity of graph neural networks and kernels does not necessarily correlate with improved generalization performance.
GBC methods compute expected utility without needing the model's density.
problem Computing expected utility in complex models.
method Density-free generative method using quantile neural estimator.
result Efficient estimation of expected utility from simulated data.
Model reveals double descent in binary linear classification.
problem Investigating classification error in high-dimensional binary linear classification.
method Gradient descent on logistic loss, maximum-likelihood, max-margin (SVM) solutions, and convex Gaussian min-max theorem.
result Double descent phenomenon observed in classification error for varying overparameterization ratio.
Graphical models trained using maximum likelihood are a common tool for probabilistic inference of marginal distributions. However, this approach suffers difficulties when either the inference process or the model is approximate. In this paper, the inference process is first defined to be the minimization of a convex f…
A new method for unsupervised domain adaptation using Gaussian processes.
problem Reducing target domain error by aligning input and output distributions.
method Max-margin Gaussian process approach to achieve hypothesis consistency.
result Our method effectively minimizes maximum discrepancy and maximizes margins.
In this paper, we present a novel and general framework called {\it Maximum Entropy Discrimination Markov Networks} (MaxEnDNet), which integrates the max-margin structured learning and Bayesian-style estimation and combines and extends their merits. Major innovations of this model include: 1) It generalizes the extant …