arXiv research

Study on VC dimension of GCNNs with input resolution effects.

Neural networks adapt to any input dimensionality.

A new method reduces both input and output dimensions for better goal-oriented analysis.

Generative adversarial networks benefit from optimal input dimension and adaptive generator architecture.

New algorithm combines new and historical data with different input dimensions for linear regression.

Study identifies key parameters and input dimensions making LLMs and VLMs brittle.

Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisi…

We study the impact of input noise dimension on GAN performance.

New $L_1$ regularization controls neural network generalization error and sparsifies input dimensions.

In this paper, we study the problem of approximately computing the product of two real matrices. In particular, we analyze a dimensionality-reduction-based approximation algorithm due to Sarlos [1], introducing the notion of nuclear rank as the ratio of the nuclear norm over the spectral norm. The presented bound has i…

Easy conditions found for simplifying complex systems.

The paper is devoted to the local classification of generic control-affine systems on an n-dimensional manifold with scalar input for any n>3 or with two inputs for n=4 and n=5, up to state-feedback transformations, preserving the affine structure. First using the Poincare series of moduli numbers we introduce the intr…

Paper introduces new bounds linking data compressibility to generalization error.

Proposes a stratified sampling method for high-dimensional models using neural active manifolds.

Test for linearizing 2-input systems with 2D feedback.

The paper analyzes why Gaussianization slows down with higher dimensions and proposes a solution.

Additive Gaussian process framework handles monotonicity constraints in high dimensions.

Random deep neural networks are robust to adversarial examples, scaling with input size and dimension.

Projection pursuit model improves Gaussian process regression for high-dimensional data.

We extend graph neural networks to transfer performance across different input sizes.

Generative models can approximate high-dimensional data from lower dimensions without needing a latent dimension equal to or greater than the data's intrinsic dimension.

Study efficient neural operator learning using variation spaces.

A typical goal of supervised dimension reduction is to find a low-dimensional subspace of the input space such that the projected input variables preserve maximal information about the output variables. The dependence maximization approach solves the supervised dimension reduction problem through maximizing a statistic…

The paper provides risk bounds for learning many response functions using linear regression.

Study on benign overfitting in leaky ReLUs with moderate input dimensions.

Develops new oracle inequalities for Gaussian ranking estimators.

This paper presents a novel decentralized high-dimensional Bayesian optimization (DEC-HBO) algorithm that, in contrast to existing HBO algorithms, can exploit the interdependent effects of various input components on the output of the unknown objective function f for boosting the BO performance and still preserve scala…

Constructs foliations from manifold group left orders.

A method to analyze neural network performance by measuring layer saturation.

Random feature matrices' singular values concentrate near their full expectation in high dimensions.

Vanilla SGD learns SIM from anisotropic data without explicit covariance estimation.

Introduces a probabilistic framework for dimension reduction methods.

New method distinguishes predictive distribution estimators in high-dimensional inputs.

Investigates the impact of finite VC dimension on neural network approximation and learning.

Deep generative networks provide a powerful tool for modeling complex data in a wide range of applications. In inverse problems that use these networks as generative priors on data, one must often perform inference of the inputs of the networks from the outputs. Inference is also required for sampling during stochastic…

Minimum width for ReLU networks to approximate L^p functions is max(d_x+1, d_y).

Reduces function approximation dimensions from high to low with sparse data.

Neural networks have been shown to be vulnerable against minor adversarial perturbations of their inputs, especially for high dimensional data under $\ell_\infty$ attacks. To combat this problem, techniques like adversarial training have been employed to obtain models which are robust on the training set. However, the …

We introduce fully scalable Gaussian processes, an implementation scheme that tackles the problem of treating a high number of training instances together with high dimensional input data. Our key idea is a representation trick over the inducing variables called subspace inducing inputs. This is combined with certain m…

We investigate the complexity of deep neural networks (DNN) that represent piecewise linear (PWL) functions. In particular, we study the number of linear regions, i.e. pieces, that a PWL function represented by a DNN can attain, both theoretically and empirically. We present (i) tighter upper and lower bounds for the m…

Deep neural networks are over-parameterized, which implies that the number of parameters are much larger than the number of samples used to train the network. Even in such a regime deep architectures do not overfit. This phenomenon is an active area of research and many theories have been proposed trying to understand …

New results show flat minima in neural networks suffer from high dimensionality.

Transformers show strengths and weaknesses in complexity analysis.

Neural networks approximate and estimate binary classifiers with polynomial input dependence.

Deep neural networks achieve optimal classification rates in high dimensions.

In this paper we propose a generalization of deep neural networks called deep function machines (DFMs). DFMs act on vector spaces of arbitrary (possibly infinite) dimension and we show that a family of DFMs are invariant to the dimension of input data; that is, the parameterization of the model does not directly hinge …

Well-established methods for the solution of stochastic partial differential equations (SPDEs) typically struggle in problems with high-dimensional inputs/outputs. Such difficulties are only amplified in large-scale applications where even a few tens of full-order model runs are impracticable. While dimensionality redu…

Random projections are able to perform dimension reduction efficiently for datasets with nonlinear low-dimensional structures. One well-known example is that random matrices embed sparse vectors into a low-dimensional subspace nearly isometrically, known as the restricted isometric property in compressed sensing. In th…