Direct optimization of discrete variational auto-encoders using arg max.
problem Optimizing discrete latent variables in variational auto-encoders.
method Direct optimization through arg max without softmax relaxations.
result Empirical effectiveness of direct loss minimization in discrete latent variables.
Entropy Search (ES) and Predictive Entropy Search (PES) are popular and empirically successful Bayesian Optimization techniques. Both rely on a compelling information-theoretic motivation, and maximize the information gained about the arg max \arg\max arg max of the unknown function; yet, both are plagued by the expensive computatio…
New RL approach builds short ancestral recombination graphs.
problem Building short ancestral recombination graphs (ARGs).
method Reinforcement Learning applied to genetic sequences.
result RL can build ARGs as short as heuristic algorithms.
NeuralSort optimizes sorting networks using continuous relaxations.
problem Non-differentiability of sorting operator hinders gradient-based optimization.
method Continuous relaxation of sorting operator to unimodal row-stochastic matrices, enabling gradient-based optimization.
result Gradient-based stochastic optimization over permutations is achieved.
We consider the problem of estimating an unknown signal x 0 x_0 x 0 from noisy linear observations y = A x 0 + z ∈ R m y = Ax_0 + z\in R^m y = A x 0 + z ∈ R m . In many practical instances, x 0 x_0 x 0 has a certain structure that can be captured by a structure inducing convex function f ( ⋅ ) f(\cdot) f ( ⋅ ) . For example, ℓ 1 \ell_1 ℓ 1 norm can be used to encourage a sparse solution. T…
Algorithm identifies the best arm in linear bandits with high probability.
problem Best arm identification in linear multi-armed bandits with noisy measurements.
method Phased Elimination Linear Exploration Game (PELEG) using no-regret learners.
result PELEG achieves sample complexity matching lower bounds.
New algorithm explores optimally using same primitives as Thompson Sampling.
problem Optimal exploration of unknown parameter vectors with noisy measurements.
method Algorithm leveraging sampling and argmax oracles.
result Achieves exponential convergence rate, optimal among all allocations.
Adaptive estimation for nonstationary time series reduces computational cost.
problem Estimating parameters of nonstationary time series with varying parameters over time.
method Moving exponential moving ML estimator for scale parameter estimation.
result Significantly improved log-likelihoods compared to standard estimation.
New algorithm identifies best arm in non-stationary linear bandits with improved complexity.
problem Best arm identification in non-stationary linear bandits with adversarial parameters.
method Proposed Adjacent-optimal design and e x t s f A d j a c e n t − B A I extsf{Adjacent-BAI} e x t s f A d j a ce n t − B A I algorithm. result Error probability matches arm-set-dependent lower bound up to constants.
Complex b b b - 6 j 6j 6 j symbols relate to hyperbolic tetrahedron volumes and determinants.
problem Analyzing asymptotics of complex b b b - 6 j 6j 6 j symbols. method Relating asymptotics to hyperbolic tetrahedron volumes and determinants.
result Complex b b b - 6 j 6j 6 j symbols' asymptotics linked to tetrahedron volumes and determinants. A novel algorithm for best-arm identification in non-stationary linear bandits reduces error probability.
problem Non-stationary environments in A/B testing scenarios.
method Proposes a novel algorithm P 1 \mathsf{P1} P1 - R A G E \mathsf{RAGE} RAGE for robust best-arm identification. result Error probability decreases as exp ( − T Δ ( 1 ) 2 / d ) \exp(-TΔ^2_{(1)}/d) exp ( − T Δ ( 1 ) 2 / d ) , demonstrating robustness to non-stationarity. LOSSGRAD automatically adjusts learning rates in neural networks.
problem Finding optimal learning rates in gradient descent.
method LOSSGRAD uses quadratic approximation to find locally optimal step-size.
result LOSSGRAD achieves comparable results to other methods while being insensitive to initial learning rate.
Given a matrix A ∈ R n × d A\in \mathbb{R}^{n\times d} A ∈ R n × d and a vector b ∈ R n b\in \mathbb{R}^n b ∈ R n , we consider the regression problem with ℓ ∞ \ell_\infty ℓ ∞ guarantees: finding a vector x ′ ∈ R d x'\in \mathbb{R}^d x ′ ∈ R d such that ∥ x ′ − x ∗ ∥ ∞ ≤ ε d ⋅ ∥ A x ∗ − b ∥ 2 ⋅ ∥ A † ∥ \|x'-x^*\|_\infty \leq \fracε{\sqrt{d}}\cdot \|Ax^*-b\|_2\cdot \|A^\dagger\| ∥ x ′ − x ∗ ∥ ∞ ≤ d ε ⋅ ∥ A x ∗ − b ∥ 2 ⋅ ∥ A † ∥ where $x^*=\arg\min_{x\in \mathbb{R}^d}\|Ax-b\|…
Study introduces statistical mechanics for min-max problems.
problem Understanding the properties of min-max problems in high dimensions.
method Statistical mechanical formalism for analyzing min-max problems.
result Derives the relationship between training data and generalization error.
Study shows strong min-max principle for phase transitions.
problem Understanding nodal sets near minimal hypersurfaces.
method Analogous to White's principle, applies to Allen-Cahn energy.
result Strong min-max principle for phase transitions.
Equity-Transformer solves NP-hard min-max routing problems efficiently.
problem Min-max routing problems with multiple agents and large-scale applications.
method Sequential planning approach with Transformer and equitable workload distribution inductive biases.
result Significant runtime and cost reductions in min-max mTSP and min-max mPDP tasks.
Study max- and min-stability under first-order stochastic dominance, finding new functional characterizations.
problem Understanding max- and min-stability in stochastic dominance.
method Representation theorem for functionals satisfying max-stability, combining max- and min-stability to define Lambda-quantiles.
result New characterizations of functionals, including Lambda-quantiles, in finance and political science.
Max-min margin Markov networks improve consistency in structured prediction.
problem Statistical inconsistency in max-margin methods for structured prediction.
method Defining a max-min margin formulation to overcome statistical inconsistency.
result Proves consistency and provides an explicit algorithm with finite sample generalization bounds.
Upper bound for Morse index of min-max varifolds.
problem Bounding Morse index of varifolds.
method Proving upper bound for Morse index of min-max stationary integral varifolds.
result Upper bound for Morse index of min-max stationary integral varifolds.
Upper bound for max-sliced 2-Wasserstein distance between measures.
problem Estimating distance between probability measures and their empirical counterparts.
method Same technique as previous work, upper bound approach.
result Upper bound for expected max-sliced 2-Wasserstein distance.
Estimates parameters in max-linear Bayesian networks with noise.
problem Causal inference in extreme-value settings with noise parameters.
method Max-plus algebra and logarithm transformation, normal distribution estimation, EM algorithm and quadratic optimization.
result An estimator of a parameter for each edge in a DAG is normally distributed.
Localized min-max method proves minimal hypersurface existence.
problem Existence of minimal hypersurfaces in complete manifolds.
method Localized min-max approach to prove existence.
result Existence of complete embedded minimal hypersurface with index at most one.
The paper solves min-max widths on a 3-sphere and strengthens multiplicity theorems.
problem Which min-max widths of the unit 3-sphere lie between 2 π 2 2π^2 2 π 2 and 8 π 8π 8 π ? method Homological min-max theory and stronger versions of multiplicity one theorems.
result Proves the 10th to 13th min-max widths of the unit 3-sphere lie between 2 π 2 2π^2 2 π 2 and 8 π 8π 8 π . Proves multiplicity one for min-max minimal hypersurfaces in specific manifolds.
problem Proving multiplicity one for min-max minimal hypersurfaces in specific manifolds.
method Using min-max theory for hypersurfaces with prescribed mean curvature and approximating min-max values.
result Confirms a conjecture by Marques-Neves for min-max minimal hypersurfaces in bumpy metrics.
Characterizes Zoll metrics via min-max values.
problem Characterizing Zoll Riemannian metrics.
method Uses min-max values in a loop space.
result Two min-max values coincide for Zoll metrics.
Max flow/min cut theorem extended to currents and topology.
problem Continuous max flow/min cut theorem for complex domains.
method Continuous analogue of max flow/min cut theorem considering topology.
result Continuous max flow/min cut theorem proven for currents and laminations.
Paper corrects Max-Margin loss for multi-label tasks.
problem Max-Margin loss inconsistency in multi-label classification.
method Introduced Restricted-Max-Margin loss.
result Consistent loss for multi-label tasks under milder conditions.
Observations depending on sums of random variables are common throughout many fields; however, no efficient solution is currently known for performing max-product inference on these sums of general discrete distributions (max-product inference can be used to obtain maximum a posteriori estimates). The limiting step to …
Adaptive momentum method solves non-convex min-max problems.
problem Non-convex min-max optimization problems in training generative adversarial networks.
method Proposes an adaptive momentum algorithm for non-convex min-max optimization.
result Establishes non-asymptotic convergence rates for the proposed algorithm.
Paper proves finiteness and Morse index estimates for equivariant min-max hypersurfaces.
problem Existence and finiteness of G G G -invariant minimal hypersurfaces. method Equivariant min-max theory, compactness theorem, bumpy metrics theorem.
result Generalization of Morse index estimates to equivariant setting.
The study bounds Morse indices of Willmore spheres in relation to min-max sweep-outs.
problem Estimating Morse indices of Willmore spheres.
method Analyzing the sum of Morse indices of Willmore spheres in min-max sweep-outs.
result At most one Willmore sphere can have index 1 among those realising min-max sphere eversion.
New findings cast doubt on the role of λ m a x λ_{max} λ ma x in generalizing neural networks.
problem The role of λ m a x λ_{max} λ ma x in neural network generalization remains unclear. method Experiments with various training interventions and batch sizes.
result Generalization benefits can vanish at larger batch sizes, challenging the role of λ m a x λ_{max} λ ma x . New proof of Smale conjecture for RP^3 and lens spaces using min-max theory.
problem Proving the Smale conjecture for specific spaces.
method Minimal surfaces and min-max theory.
result New proof of Smale conjecture for R P 3 \mathbb{RP}^3 RP 3 and lens spaces. Paper improves Morse index bound for hypersurfaces.
problem Improving Morse index bound for hypersurfaces.
method Construction of hierarchical deformations and restrictive min-max theory.
result Generalizes a result by X. Zhou for 3 ≤ n + 1 ≤ 7 3 \leq n+1 \leq 7 3 ≤ n + 1 ≤ 7 . We consider the generic regularized optimization problem β ^ ( λ ) = arg min β L ( y , X β ) + λ J ( β ) \hat{\mathsfβ}(λ)=\arg \min_βL({\sf{y}},X{\sfβ})+λJ({\sfβ}) β ^ ( λ ) = arg min β L ( y , X β ) + λ J ( β ) . Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407--499] have shown that for the LASSO--that is, if L L L is squared error loss and J ( β ) = ∥ β ∥ 1 J(β)=\|β\|_1 J ( β ) = ∥ β ∥ 1 is the ℓ 1 \ell_1 ℓ 1 norm of β β β --the opti…
New methods solve min-max problems on manifolds using Riemannian Hamiltonians.
problem Min-max optimization on Riemannian manifolds.
method Riemannian Hamiltonian methods (RHM) to minimize the Hamiltonian function.
result RHM leads to correct search directions and global optimality in min-max problems.
Tropical geometry and weighted lattices improve curve and surface fitting.
problem Fitting max- ⋆ \star ⋆ tropical curves and surfaces to data. method Max- ⋆ \star ⋆ algebra, weighted lattices, morphological adjunctions. result Optimal piecewise-linear regression for max- ⋆ \star ⋆ curves and surfaces. Max-plus operators improve neural network filter selection and pruning.
problem Improving neural network efficiency and reducing redundancy.
method Exploiting Max-plus operators in neural network layers for filter selection and model pruning.
result Max-plus layers enhance filter selection and reduce redundancy without performance loss.
Bound on equivariant index for min-max surfaces.
problem Bounding the index of equivariant min-max surfaces.
method Equivariant min-max procedure with group action.
result Equivariant index bound by number of parameters.
Survey of advances in non-convex min-max optimization for applications.
problem Finding optimal solutions in non-convex, non-concave min-max problems.
method Selective review of theoretical and algorithmic advances.
result Exciting recent advances in solving non-convex min-max problems.
Max-affine regression method converges linearly using GD and SGD.
problem Regression of max-affine models in signal processing and statistics.
method Gradient descent and mini-batch stochastic gradient descent analysis.
result GD and SGD converge linearly to a neighborhood of the ground truth under sub-Gaussian assumptions.
Investigates max width of unit volume 3-spheres in conformal classes.
problem Maximizing width of 3-spheres with fixed volume and conformal class.
method Simon-Smith min-max theory applied to Riemannian 3-spheres.
result Characterizes maximising metrics and how they change with conformal class.
We reformulate LIPs as min-max problems for easier solution.
problem Recovering signals from few linear measurements.
method Proposed a min-max reformulation of LIPs.
result Saddle points characterize solutions to LIPs.
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Epoch-GDA achieves optimal convergence rate for SCSC min-max problems.
problem Solving stochastic min-max problems with strong convexity and strong concavity.
method Epoch-wise stochastic gradient descent ascent method (Epoch-GDA) without additional assumptions.
result Achieves the optimal rate of O ( 1 / T ) O(1/T) O ( 1/ T ) for the duality gap of general SCSC min-max problems. The paper bounds the min-max width of embedded circles on spheres and manifolds.
problem Bounding the min-max width of embedded circles on spheres and manifolds.
method Inducing a sweepout by pairs of points in embedded circles from a given sweepout of the sphere by closed curves.
result Lower bounds for the Birkhoff min-max invariant of a Riemannian sphere in terms of the min-max width of its embedded circles.
Study confirms a 2-sphere metric with three geodesics of minimal length.
problem Understanding the systolic, width, and Gromov-Guth metrics on a 2-sphere.
method Classical min-max and hyperbolic geometry tools.
result Figure-eight geodesics achieve the systolic, width, and Gromov-Guth metrics on a 2-sphere.
This paper improves MDS visualization by adjusting Wasserstein distances for heavy-tailed data.
problem Enhancing Multidimensional Scaling (MDS) for better pattern recognition with heavy-tailed distributions.
method Introduces Max-D-SW, a metric adjustment of Max-Sliced Wasserstein distance that aggregates over orthonormal bases.
result Max-D-SW provides a clear numerical advantage in MDS outcomes, especially for heavy-tailed distributions.