arXiv research

A new algorithm tackles bilevel optimization with multiple inner minima.

Paper proposes a new method to optimize feature coordinates for better image classification.

A core capability of intelligent systems is the ability to quickly learn new tasks by drawing on prior experience. Gradient (or optimization) based meta-learning has recently emerged as an effective approach for few-shot learning. In this formulation, meta-parameters are learned in the outer loop, while task-specific m…

Study of Gaussian distributions using entropic Gromov-Wasserstein and inner product Gromov-Wasserstein.

A new stochastic method tackles bi-level optimization problems in deep learning.

Convex learning for diverse invariances in semi-inner-product space.

New approach to bilevel optimization for machine learning using functional methods.

Teaches sequential learners with changing inner states to improve future performance.

Paper improves adversarial training using a learned optimizer.

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

Optimizes shapes of curves using Möbius energy gradients.

Fewer data weight updates lead to faster convergence in machine learning models.

MAML optimizes shared priors for subtasks in a nonconvex meta-objective.

Bayesian optimization uses triangulation candidates for better performance.

New algorithm for linear bandits tackles Optimal Transport problems.

Mathematical optimization is widely used in various research fields. With a carefully-designed objective function, mathematical optimization can be quite helpful in solving many problems. However, objective functions are usually hand-crafted and designing a good one can be quite challenging. In this paper, we propose a…

A method for fair representation learning through bi-level optimization and implicit differentiation.

Equivalent tests for SGD batch size selection found.

This paper explores the non-convex composition optimization in the form including inner and outer finite-sum functions with a large number of component functions. This problem arises in some important applications such as nonlinear embedding and reinforcement learning. Although existing approaches such as stochastic gr…

RSGDA improves convergence rates for nonconvex-strongly concave optimization.

We study estimation of (semi-)inner products between two nonparametric probability distributions, given IID samples from each distribution. These products include relatively well-studied classical $\mathcal{L}^2$ and Sobolev inner products, as well as those induced by translation-invariant reproducing kernels, for whic…

We propose a robust elastic net (REN) model for high-dimensional sparse regression and give its performance guarantees (both the statistical error bound and the optimization bound). A simple idea of trimming the inner product is applied to the elastic net model. Specifically, we robustify the covariance matrix by trimm…

The paper proposes an efficient nested simulation design using likelihood ratio method.

The variance reduction class of algorithms including the representative ones, SVRG and SARAH, have well documented merits for empirical risk minimization problems. However, they require grid search to tune parameters (step size and the number of iterations per inner loop) for optimal performance. This work introduces `…

We propose $\textit{weighted inner product similarity}$ (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities in…

Neural network pruning reduces the computational cost of an over-parameterized network to improve its efficiency. Popular methods vary from $\ell_1$-norm sparsification to Neural Architecture Search (NAS). In this work, we propose a novel pruning method that optimizes the final accuracy of the pruned network and distil…

We consider a class of a nested optimization problems involving inner and outer objectives. We observe that by taking into explicit account the optimization dynamics for the inner objective it is possible to derive a general framework that unifies gradient-based hyperparameter optimization and meta-learning (or learnin…

DCGD improves training of PINNs by adjusting gradients to avoid negative inner products.

New method solves complex optimization problems with real-time learning.

Optimizes K inner simulations for least-square Monte Carlo to reduce computational cost.

ES-Single uses ES to estimate gradients in unrolled graphs, reducing variance and improving performance.

Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix factorization approximates the entries of the matrix by a simple, fixed function-…

NHGD solves bilevel optimization problems with reduced computational time.

This thesis analyzes and improves convergence rates of bilevel optimization algorithms in machine learning.

New method solves complex constrained optimization problems.

Amortizes MIPS by training neural networks to predict optimal keys.

Study higher rank inner products and their tilings to describe tori degenerations.

Characterizes quandles with abelian inner automorphisms.

New spectral functionals for Dirac operators with inner fluctuations computed.

In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations. Unlike other variance reduction techniques that either require additional storage or the regular computation of full gradients, the proposed method reduces variance by …

This paper constructs quandles with abelian inner automorphism groups from graphs, proving their homogeneity.

Improved penalty-based methods for bilevel optimization with reduced complexity.

Groups with specific properties have vanishing $\ell^2$-Betti numbers.

Considering mean-variance portfolio problems with uncertain model parameters, we contrast the classical absolute robust optimization approach with the relative robust approach based on a maximum regret function. Although the latter problems are NP-hard in general, we show that tractable inner and outer approximations e…

The author reviews his results on locally compact homogeneous spaces with inner metric, in particular, homogeneous manifolds with inner metric. The latter are isometric to homogeneous (sub-)Finslerian manifolds; under some additional conditions they are isometric to homogeneous (sub)-Riemannian manifolds. The class $Ω$…

Researchers prove inner product recovery is impossible in latent space models.

PDA method optimizes neural networks with global convergence rate analysis.

Enhances CLIP's similarity computation using PMI's linear structure.