arXiv research

A new method reduces both input and output dimensions for better goal-oriented analysis.

Generative adversarial networks benefit from optimal input dimension and adaptive generator architecture.

We study the impact of input noise dimension on GAN performance.

This paper presents a novel decentralized high-dimensional Bayesian optimization (DEC-HBO) algorithm that, in contrast to existing HBO algorithms, can exploit the interdependent effects of various input components on the output of the unknown objective function f for boosting the BO performance and still preserve scala…

Deep neural networks achieve optimal classification rates in high dimensions.

New $L_1$ regularization controls neural network generalization error and sparsifies input dimensions.

Paper reduces hyperparameters in mixed-categorical Gaussian processes for green aircraft optimization.

Zeroth-order optimization is the process of minimizing an objective $f(x)$, given oracle access to evaluations at adaptively chosen inputs $x$. In this paper, we present two simple yet powerful GradientLess Descent (GLD) algorithms that do not rely on an underlying gradient estimate and are numerically stable. We analy…

Paper introduces new bounds linking data compressibility to generalization error.

Study on VC dimension of GCNNs with input resolution effects.

Recommendation problems with large numbers of discrete items, such as products, webpages, or videos, are ubiquitous in the technology industry. Deep neural networks are being increasingly used for these recommendation problems. These models use embeddings to represent discrete items as continuous vectors, and the vocab…

The paper is devoted to the local classification of generic control-affine systems on an n-dimensional manifold with scalar input for any n>3 or with two inputs for n=4 and n=5, up to state-feedback transformations, preserving the affine structure. First using the Poincare series of moduli numbers we introduce the intr…

RBM learns in high dimensions via AMP and GD, reaching optimal weak recovery.

Neural networks adapt to any input dimensionality.

Bayesian optimization improves with nonstationary covariance functions.

KOLMOGOROV-OPTIMAL RESOLUTION ESTIMATION (KORE) solves spline regression without exhaustive search

We solve the problem of minimizing the number of critical points among all functions on a surface within a prescribed distance δ from a given input function. The result is achieved by establishing a connection between discrete Morse theory and persistent homology. Our method completely removes homological noise with pe…

This article introduces the concepts around Online Bandit Linear Optimization and explores an efficient setup called SCRiBLe (Self-Concordant Regularization in Bandit Learning) created by Abernethy et. al.\cite{abernethy}. The SCRiBLe setup and algorithm yield a $O(\sqrt{T})$ regret bound and polynomial run time comple…

The sparse pseudo-input Gaussian process (SPGP) is a new approximation method for speeding up GP regression in the case of a large number of data points N. The approximation is controlled by the gradient optimization of a small set of M `pseudo-inputs', thereby reducing complexity from N^3 to NM^2. One limitation of th…

We consider the problem of online adaptive control of the linear quadratic regulator, where the true system parameters are unknown. We prove new upper and lower bounds demonstrating that the optimal regret scales as $\widetildeΘ({\sqrt{d_{\mathbf{u}}^2 d_{\mathbf{x}} T}})$, where $T$ is the number of time steps, $d_{\m…

Improved Thompson Sampling outperforms existing Bayesian optimization methods.

Optimal hashing embeddings reduce linear least squares solving time.

t-SNE is a popular tool for embedding multi-dimensional datasets into two or three dimensions. However, it has a large computational cost, especially when the input data has many dimensions. Many use t-SNE to embed the output of a neural network, which is generally of much lower dimension than the original data. This l…

For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify its statistics, using the minimum number of function evaluations. This problem …

Metalearning optimizes autoencoder dimensions for efficient data representation.

Deep generative networks provide a powerful tool for modeling complex data in a wide range of applications. In inverse problems that use these networks as generative priors on data, one must often perform inference of the inputs of the networks from the outputs. Inference is also required for sampling during stochastic…

In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared loss of such CNNs is $\mathit{~locally~strongly~convex}$ in a basin of attraction…

Bayesian optimization improves by focusing on outputs with the likelihood ratio method.

High-dimensional kernel regression struggles due to rotational invariance.

New algorithm combines new and historical data with different input dimensions for linear regression.

New bounds for adaptive control in high dimensions without fixed state space.

A method for predicting signals on graphs using Gaussian processes and optimal transport.

Optimizes functionals on probability space using ICNNs.

Study identifies key parameters and input dimensions making LLMs and VLMs brittle.

The paper explores how to reduce classification tasks to optimization problems in Euclidean space.

Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisi…

Proves depth 2 neural networks can't approximate certain functions as well as depth 3 networks.

RCNPs extend equivariant neural processes to higher dimensions, improving performance on tasks with inherent symmetries.

Bayes-optimal learning of deep random networks with Gaussian weights is studied.

We present an approximation scheme for support vector machine models that use an RBF kernel. A second-order Maclaurin series approximation is used for exponentials of inner products between support vectors and test instances. The approximation is applicable to all kernel methods featuring sums of kernel evaluations and…

New method optimizes expensive simulations for complex systems.

Improved stochastic gradient estimation for deep learning in high dimensions.

We consider in this paper the regularity problem for time-optimal trajectories of a single-input control-affine system on a n-dimensional manifold. We prove that, under generic conditions on the drift and the controlled vector field, any control u associated with an optimal trajectory is smooth out of a countable set o…

Algorithm finds optimal regularizers for online linear optimization.

Study reveals phase transition in neural networks near interpolation.

Optimal sample complexity for contrastive learning of distances.

A new model predicts multivariate regression using similarities to data points.

Wide neural networks converge linearly to zero loss with feature learning.