We study the problem of alleviating the instability issue in the GAN training procedure via new architecture design. The discrepancy between the minimax and maximin objective values could serve as a proxy for the difficulties that the alternating gradient descent encounters in the optimization of GANs. In this work, we…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Minimax architecture”
A new method trains physics-constrained neural networks more efficiently.
problem Training machine learning tools with limited data and physical constraints.
method Dual-Dimer method for searching saddle points in nonconvex-nonconcave functions.
result The Dual-Dimer method improves training efficiency and convergence speed.
Study minimax rates for density estimation under Huber contamination and Besov IPM losses.
problem Minimax convergence rates of nonparametric density estimation under Huber contamination model with outliers.
method Re-scaled thresholding wavelet series estimator and GAN architectures.
result Achieves minimax optimal convergence rates under Besov IPM losses.
Deep neural network is a state-of-art method in modern science and technology. Much statistical literature have been devoted to understanding its performance in nonparametric estimation, whereas the results are suboptimal due to a redundant logarithmic sacrifice. In this paper, we show that such log-factors are not nec…
Measuring the similarity of two files is an important task in malware analysis, with fuzzy hash functions being a popular approach. Traditional fuzzy hash functions are data agnostic: they do not learn from a particular dataset how to determine similarity; their behavior is fixed across all datasets. In this paper, we …
PAC-Bayesian bounds show fully connected DNNs with Gaussian priors match minimax rates.
problem Theoretical limits of fully connected deep neural networks with Gaussian priors.
method PAC-Bayesian bounds for fully connected Bayesian DNNs with Gaussian priors.
result PAC-Bayesian bounds match minimax-optimal rates in Besov space for nonparametric regression and binary classification.
We describe a novel extension of soft actor-critics for hierarchical Deep Q-Networks (HDQN) architectures using mutual information metric. The proposed extension provides a suitable framework for encouraging explorations in such hierarchical networks. A natural utilization of this framework is an adversarial setting, w…
Generalization in deep neural networks can be analyzed using minimax rates for gradient methods.
problem Generalization performance of over-parameterized neural networks
method Establishing a connection between gradient-based methods and kernel methods
result Deriving minimax-optimal rates for GD and SGD under polynomial network width scaling
Diffusion models adapt to low-dimensional structures for nonparametric density estimation.
problem High-dimensional statistical inference challenges.
method Viewing diffusion models as implicit density estimators and exploiting their low-dimensional structure.
result Achieves minimax optimal rate for total variation distance with factorizable density.
Deep learning models can adaptively estimate functions with varying smoothness using regularization.
problem Estimating functions with heterogeneous smoothness in Besov or BV classes.
method Introduced a Parallel NN variant of deep ReLU networks with regularization equivalent to promoting -sparsity.
result Achieves minimax rates for Besov and BV classes with exponentially closer performance as depth increases.
We consider the problem of training generative models with a Generative Adversarial Network (GAN). Although GANs can accurately model complex distributions, they are known to be difficult to train due to instabilities caused by a difficult minimax optimization problem. In this paper, we view the problem of training GAN…
Neural networks outperform NTK on compositional tasks, revealing a complexity gap.
problem Understanding the performance gap between neural networks and NTK on tasks with compositional structure.
method Characterized Fourier and architectural complexities, and analyzed the minimax rates of the architecture class.
result The NTK estimator is exponentially sub-optimal compared to the minimax floor when complexities decouple.
Paper proposes a new method for regression using optimal transport cost optimization.
problem Regression problem, especially with complex noise forms.
method Directly optimizes the optimal transport cost between true and estimated distributions.
result Achieves state-of-the-art results on various datasets.
GAT-GMM improves GANs' performance in learning Gaussian mixture models.
problem GANs struggle with multi-modal distributions like Gaussian mixtures.
method Proposes a minimax GAN framework using random linear generator and softmax-based quadratic discriminator.
result Gradient Descent Ascent method converges to an approximate minimax point.
We deconstruct the performance of GANs into three components: 1. Formulation: we propose a perturbation view of the population target of GANs. Building on this interpretation, we show that GANs can be viewed as a generalization of the robust statistics framework, and propose a novel GAN architecture, termed as Cascade …
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscs.LG
SNGP improves DNNs' uncertainty estimation with minimal changes.
problem Uncertainty estimation in deep learning models for real-time applications.
method Formalizing uncertainty as a minimax problem, SNGP adds weight normalization and replaces the output layer with a Gaussian process.
result SNGP outperforms other single-model approaches in uncertainty estimation across vision and language tasks.
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to -factors) under a general composition assumption on the re…
New BNN model proves optimal posterior concentration and enables practical inference.
problem Improving generalization and uncertainty quantification in deep neural networks.
method Proposes a new node-sparse BNN model with theoretical guarantees and a novel MCMC algorithm for inference.
result Proves near minimax optimal posterior concentration rate and adaptiveness to true model smoothness.
Gradient descent methods for deep ReLU networks achieve optimal generalization rates.
problem Generalization of gradient descent methods for deep neural networks
method Establishing minimax-optimal rates for GD and SGD with deep ReLU networks
result Gradient descent methods for deep ReLU networks achieve optimal generalization rates
Two new algorithms solve nonconvex-strongly concave problems efficiently.
problem Solving nonconvex-strongly concave minimax problems.
method Proposed MINIMAX-TR and MINIMAX-TRACE algorithms.
result Find -second order stationary points within iterations.
A novel decentralized algorithm improves minimax optimization in federated learning.
problem Minimax optimization in federated learning with data heterogeneity.
method Decentralized Gradient Tracking (K-GT-Minimax) for nonconvex-strongly-concave optimization.
result Demonstrates superior convergence rate for NC-SC minimax optimization.
Near-optimal rates for multi-task learning with shared representations.
problem Approximation and statistical complexity of learning multiple operators.
method Multiple Neural Operators (MNO) architecture and comparison with DeepONet.
result Near-optimal upper and lower bounds for approximation and generalization.
Any autoencoder network can be turned into a generative model by imposing an arbitrary prior distribution on its hidden code vector. Variational Autoencoder (VAE) [2] uses a KL divergence penalty to impose the prior, whereas Adversarial Autoencoder (AAE) [1] uses {\it generative adversarial networks} GAN [3]. GAN trade…
We investigate the use of Minimax distances to extract in a nonparametric way the features that capture the unknown underlying patterns and structures in the data. We develop a general-purpose and computationally efficient framework to employ Minimax distances with many machine learning methods that perform on numerica…
Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spacesstat.ML
Bayesian KANs achieve near-minimax posterior contraction rates in anisotropic Besov spaces.
problem Statistical foundation for Bayesian Kolmogorov-Arnold networks in anisotropic Besov spaces.
method Sparse Bayesian KANs with spike-and-slab priors, hyperprior on model size, and approximation complexity bounds.
result Posterior contraction rates depend on intrinsic anisotropic smoothness and effective dimension of the compositional structure.
Many tasks in modern machine learning can be formulated as finding equilibria in \emph{sequential} games. In particular, two-player zero-sum sequential games, also known as minimax optimization, have received growing interest. It is tempting to apply gradient descent to solve minimax optimization given its popularity a…
Bayesian deep learning with heavy-tailed weights achieves near-optimal performance.
problem Deep neural networks with heavy-tailed weights achieve near-optimal performance in various contexts.
method Introduced a Bayesian deep learning prior based on heavy-tailed weights and ReLU activation, showing near-optimal minimax contraction rates.
result Posterior distribution achieves near-optimal minimax contraction rates, adaptive to smoothness and intrinsic dimension.
This paper analyzes saddle points and minimax points in non-convex smooth games.
problem Understanding local optimal points in non-convex smooth games.
method Comprehensive analysis of local minimax points, including their optimality conditions and stability.
result Local saddle points are uniformly local minimax points under mild continuity assumptions.
Paper explores generalization of minimax learners, proposing a new metric.
problem Understanding how minimax learners perform on unseen data.
method Proposes a new metric, the primal gap, to study generalization of minimax learners.
result Derives generalization error bounds for the primal gap in nonconvex-concave settings.
A new method solves complex constrained minimax problems.
problem Solving constrained minimax optimization problems.
method First-order augmented Lagrangian method.
result Established an operation complexity of .
T-Cal tests model calibration with a minimax optimal test.
problem Detecting mis-calibration of predictive models using a finite validation dataset.
method T-Cal is a minimax optimal test for calibration based on a debiased plug-in estimator of the -Expected Calibration Error (ECE).
result T-Cal is a practical tool for testing the calibration of probabilistic classification methods.
New algorithm solves federated minimax optimization problems.
problem Federated minimax optimization challenges.
method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).
result FESS-GDA uniformly solves federated minimax problems.
Stochastic descent methods (of the gradient and mirror varieties) have become increasingly popular in optimization. In fact, it is now widely recognized that the success of deep learning is not only due to the special deep architecture of the models, but also due to the behavior of the stochastic descent methods used, …
Paper proposes an algorithm to solve complex minimax problems efficiently.
problem Stochastic nonconvex-concave minimax problems in various fields.
method Accelerated first-order regularized momentum descent ascent algorithm (FORMDA).
result Achieves best-known complexity bound of for single-loop algorithms.
A new decentralized method solves minimax problems with reduced communication and sample complexity.
problem Solving minimax optimization problems in a distributed setting.
method Decentralized stochastic gradient descent ascent with variance reduction.
result Achieved optimal sample and communication complexities for nonconvex-strongly-concave problems.
Paper proposes a new sparse Bayesian neural network for simpler, more efficient DNNs.
problem Complex and large DNN architectures require simplification for better performance and efficiency.
method Masked Bayesian Neural Networks (BNN) with nodewise sparsity and optimal posterior distributions.
result The proposed BNN yields well-condensed DNN architectures with similar accuracy and uncertainty quantification to large DNNs.
Efficient sampling reduces memory usage for Minimax distance analysis.
problem Quadratic memory requirement for existing Minimax distance methods.
method Proposes a novel sampling technique with linear space complexity.
result Demonstrates significant reduction in memory usage for Minimax distances.
Study on estimating invertible functions with minimax analysis.
problem Minimizing risk of estimating invertible functions on a plane.
method Introduce two types of -risks, derive lower and upper rates for minimax values, develop an asymptotically almost everywhere invertible estimator.
result Invertibility does not reduce the complexity of the estimation problem in terms of the rate.
Minimax defense improves neural network security against gradient-based attacks.
problem Gradient-based adversarial attacks on neural networks.
method Minimax optimization in a GAN framework to create a discriminator that plays a minimax game with the generator.
result Minimax defense significantly reduces adversarial attack success rates compared to standard classifiers.
Benchmarking Minimax Linkagestat.ML
Minimax linkage was first introduced by Ao et al. [3] in 2004, as an alternative to standard linkage methods used in hierarchical clustering. Minimax linkage relies on distances to a prototype for each cluster; this prototype can be thought of as a representative object in the cluster, hence improving the interpretabil…
New bounds on minimax regret for sequential probability assignment using logarithmic loss.
problem Minimizing regret in sequential probability assignment against arbitrary experts.
method Using self-concordance property of logarithmic loss to derive tight bounds.
result Tight bounds on minimax regret for various expert classes.
Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.
problem Nonconvex-nonconcave minimax problems with PL condition.
method Zeroth-order AGDA and VRAGDA algorithms.
result Iteration complexities for obtaining ε-stationary points.
New algorithm AG-OG optimizes separable convex-concave problems efficiently.
problem Efficiently solving separable convex-concave minimax optimization problems.
method Leverages Nesterov acceleration and optimistic gradient on component and coupling parts of the problem.
result Achieves optimal convergence rate for various settings including bilinearly coupled problems.
Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.
problem Nonconvex-concave minimax optimization problems.
method Two shuffling gradient-based algorithms for nonconvex-linear and nonconvex-strongly concave settings.
result Achieves state-of-the-art oracle complexity in nonconvex optimization and best-known complexity bounds for nonconvex-strongly concave setting.
Paper improves risk bounds for nonconvex-strongly-concave minimax problems.
problem Achieving sharper risk bounds for nonconvex-strongly-concave minimax problems.
method Using uniform localized convergence to derive high probability generalization error bounds.
result Derives n times faster excess primal risk bounds for popular algorithms.
Optimally estimate distances on surfaces using reconstructed meshes.
problem Estimating intrinsic distances on smooth submanifolds.
method Reconstruction of the surface using a tangential Delaunay complex, and Isomap variant.
result Minimax optimality achieved for distance estimation.
Study minimax linear regression under quantile risk, improving existing bounds and providing new results.
problem Designing minimax procedures in linear regression under quantile risk.
method Analyzes realizable setting with Gaussian noise, extends to all p-th power error functions, develops new lower and upper bounds.
result Proves minimaxity of a variant of the min-max regression procedure for all p-th power error functions.
The paper analyzes generalization bounds for NC-SC/NC-C stochastic minimax optimization.
problem Generalization analysis of nonconvex-(strongly)-concave stochastic minimax optimization.
method Established algorithm-agnostic and algorithm-dependent generalization bounds via uniform convergence and stability arguments.
result Sample complexities and generalization bounds for NC-SC and NC-C settings.